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EXECUTIVE  SUMMARY 


The  purpose  of  this  research  effort  was  to  demonstrate  the  benefits  of  using  sophisticated  batch 
classification  methods  to  improve  the  quality  of  enlisted  Military  personnel.  In  order  to  reach  the 
objective,  this  project  used  the  Belgian  Psychometric  Model  as  a  technology  demonstrator.  The  aim 
was  to  use  this  system  with  actual  selection  data  from  different  countries  to  show  that  it  can 
improve  the  quality  of  the  enlisted  persons  through  smart  classification.  In  this  report  we  review 
three  datasets  originating  from  respectively  Portugal,  Spain  and  the  United  States. 

The  table  below  illustrates  the  core  of  what  is  meant  by  the  title  of  this  research  project.  The  two 
compared  classification  methods  deal  with  the  same  applicants  for  the  same  vacancies.  Both 
methods  fill  the  vacancies.  Yet,  the  available  indicators  of  the  Military  recruit  quality  (in  this 
example,  the  relevant  composite  scores)  are  significantly  better  when  the  classification  is  done  by  a 
smart  classification  system  such  as  the  Psychometric  Model. 


Composite 

USAF 

Assignment 

Psychometric  Model 
Assignment 

Difference  in 
average 

#  observations 

Average 

#  observations 

Average 

Mechanical 

7826 

69.19 

7826 

73.80 

+4,6 

General 

10608 

62.75 

10608 

68.82 

+6,06 

Electronics 

6050 

73.57 

5924 

77.42 

+3,85 

Admin 

3587 

69.99 

3587 

85.16 

+15,17 

Total 

28071 

27945 

The  benefits  of  using  such  a  method  should  be  estimated  by  comparing  the  costs  of  implementing  it 
versus  the  costs  of  alternative  methods  needed  to  reach  a  similar  quality  increase,  such  as  increased 
advertising,  recruitment  bonuses  etc.  Given  the  marginal  costs  of  smart  classification,  there  is  little 
doubt  that  using  such  a  method  is  the  easiest  way  to  augment  recruit  quality. 

Due  to  the  impressive  number  of  parameters  involved  in  classification  questions,  it  is  quite  difficult 
to  model  the  outcome  of  particular  classification  settings.  That  is,  unless  very  elementary  and,  as  a 
consequence,  much  less  powerful  classification  methods  are  used.  It  is  therefore  necessary  to  assess 
the  power  of  a  classification  method  by  testing  it  on  a  wide  variety  of  classification  problems  and 
to  compare  its  outcome  to  that  of  alternative  methods.  That  is  what  was  done  with  this  research 
project.  Based  upon  these  particular  datasets,  we  can  conclude  that  the  Psychometric  Model  did  at 
least  as  well  as  the  different  methods  it  was  compared  to.  In  addition,  it  showed  to  be  able  to  assign 
more  people  to  jobs  without  decreasing  the  average  payoff  significantly  or  respecting  the 
applicants’  preferences  less. 

Future  research  should  be  carried  out  to  increase  the  number  of  datasets  used  to  demonstrate  the 
benefits  of  smart  classification.  It  is  recommended  that  such  datasets  include  all  eligible  applicants 
and  not  only  those  who  got  a  job  in  the  original  situation  and  that  information  concerning  their 
preferences  towards  the  entries  would  be  included  as  well. 

Classification  methodologies  deal  with  variables  and  business  rules  that  are  specific  for  particular 
countries  or  classification  settings.  Yet,  the  methodologies  can  easily  be  implemented  in  different 
countries  or  settings.  Military  recruitment  settings  are  different  to  many  civilian  recruitment 
problems  where  multiple-job  environments  are  more  the  exception  than  the  rule.  Classification 
technology  is  for  these  two  reasons  a  topic  that  is  very  well  suited  for  international  military 
collaboration.  It  is  therefore  hoped  that  the  research  effort  made  possible  by  the  US  Office  of  Naval 
Research  can  be  sustained  in  the  future. 
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1.  Introduction 

In  times  where  recruiting  objectives  are  hard  to  achieve,  it  is  temping  to  lower  entry 
standards  in  order  to  increase  the  proportion  of  applicants  that  are  eligible  for  enlistment. 
This  however,  decreases  the  overall  quality  of  the  accepted  recruits.  By  recognizing  the 
distribution  of  aptitudes  among  the  group  of  applicants  along  with  the  distribution  of 
required  aptitudes  to  qualify  for  the  different  vacancies,  it  becomes  possible  to  improve 
the  overall  quality  of  enlisted  recruits  using  smart  allocation  methods.  Put  in  other  words, 
optimizing  person-job  match,  not  only  at  the  individual  but  also  at  the  group  level,  allows 
to  better  capitalizing  on  the  qualities  available  in  the  pool  of  applicants.  Improved  recruit 
quality  has  tremendous  positive  consequences  to  the  Military  community  in  general. 
Obtaining  improved  recruit  quality  at  no  extra  costs  using  smart  classification  is  therefore 
a  goal  worthwhile  pursuing. 

The  purpose  of  the  described  research  effort  was  to  demonstrate  the  benefits  of  using 
sophisticated  batch  classification  methods  to  improve  the  quality  of  enlisted  Military 
personnel. 

In  order  to  reach  the  objective,  this  project  used  the  Belgian  Psychometric  Model  as  a 
technology  demonstrator.  The  aim  was  to  use  the  Psychometric  Model  with  actual 
selection  data  from  different  countries  to  show  that  it  can  improve  the  quality  of  the 
enlisted  persons  through  smart  allocation.  A  prerequisite  of  this  approach  is  that 
recruitment  encompasses  vacancies  requiring  differential  aptitudes. 

2.  Background 

This  research  effort  was  initiated  following  the  request  made  by  Craig  Dorman  (Office  of 
Naval  Research  (ONR)  at  the  International  Applied  Military  Psychology  Symposium 
(IAMPS)  in  Split,  Croatia,  in  September  2000.  Craig  Dorman  quoted  Admiral  Vern 
Clark,  US  Chief  of  Naval  Operations,  who  said:  ‘My  top  1  priority  is  recruiting  Sailors, 
retaining  Sailors  and  fighting  the  attrition  of  Sailors.’  (Assuming  the  Watch,  July  2000). 
He  asked  the  IAMPS  community  to  propose  international  collaborative  research  projects 
aimed  at  fighting  the  current  military  recruitment  and  retention  problems  experienced  by 
many  countries.  The  present  research  project  was  awarded  at  the  IAMPS  ‘International 
Workshop  on  Military  Recruitment  and  Retention’  held  in  The  Hague,  The  Netherlands, 
in  April  2001.  Its  completion  was  made  possible  by  the  sponsorship  of  the  US  Office  of 
Naval  Research  and  the  cooperation  of  dedicated  colleagues  from  different  countries  who 
provided  us  with  selection  and  classification  datasets. 


1  To  contact  the  author:  e-mail:  lescreve@skvnet.be  or  Francois.Lescreve@mil.be  ,  Phone:  +32  2  264  5751,  Fax:  +32  2  264  5729 
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3.  Method 

a.  The  Psychometric  Model 

The  Psychometric  Model  is  a  smart  classification  tool  developed  by  the  author  and  his 
team  and  used  for  the  classification  of  the  Belgian  NCOs  since  1995.  Although  it 
would  bring  us  too  far  to  describe  the  tool  exhaustively  in  this  paper2,  some 
comprehension  of  its  principles  is  necessary  for  understanding  the  research  results. 
The  tool  is  meant  for  batch  classification  and  requires  a  pool  of  applicants  and  a  set  of 
jobs.  Identical  jobs  are  clustered  in  ‘trades’  or  ‘entries’.  Each  trade  can  contain  several 
identical  jobs.  On  the  applicant’s  side,  the  Psychometric  Model  uses  selection 
variables  (both  metric  and  categorical)  and  preferences  towards  the  different  trades. 
For  each  entry,  a  minimum  and/or  maximum  value  can  be  set  for  each  metric  variable 
together  with  a  weight.  That  weight  can  be  compared  to  a  beta  weight  used  in  multiple 
linear  regressions.  An  initial  payoff  is  computed  for  each  applicant-trade  combination 
as  a  weighted  sum  using  the  applicant’s  standardized  metric  variable  scores  and  the 
weights  given  to  those  variables  for  the  trade.  Applicants  not  meeting  the  set  minima 
or  maxima  for  a  trade  are  rejected  for  that  particular  trade. 

For  each  entry,  for  each  categorical  variable,  a  coefficient  can  be  set  for  each  class. 
Such  a  coefficient  will  then  be  used  multiplicatively  to  adapt  the  initial  payoff. 
Coefficients  larger  than  one  will  be  set  for  classes  that  are  highly  appropriate  for  the 
considered  entry.  For  instance,  a  person  who  took  training  in  mechanics  (and  belongs 
to  that  class)  could  get  an  increase  of  his  payoff  of  10%  (coefficient  =  1.1)  for  a  trade 
such  as  ‘Vehicle  &  Equipment  Maintenance  Apprentice’.  Coefficients  smaller  than 
one  will  have  the  opposite  effect.  Coefficients  zero  will  reject  an  applicant  for  the 
particular  entry.  For  instance,  a  candidate  who  hasn’t  a  specific  security  clearance  (and 
thus  belonging  to  a  class  ‘no  clearance’  on  a  variable  ‘security  status’)  can  be  rejected 
for  an  entry  like  ‘Signals  Intelligence  Analysis  Apprentice’  by  setting  the  coefficient 
for  this  combination  to  zero. 

In  a  third  step,  the  payoffs  will  be  adapted  again  according  to  the  applicant’s 
preference  towards  the  trade  and  the  importance  that  is  given  to  respecting  the  choices 
of  the  applicants.  After  that  third  step,  we  dispose  of  payoff  values  that  represent  the 
utility  of  assigning  the  applicant  to  a  particular  trade. 

In  the  last  step,  we  put  all  these  payoff  values  in  a  matrix  containing  the  applicants  as 
row  headers  and  the  jobs  as  column  headers.  The  matrix  is  squared  by  adding  dummy 
jobs  (or  dummy  persons)  with  payoff  values  set  to  zero.  Then,  by  using  an  operational 
research  algorithm,  we  link  each  applicant  to  a  job  in  such  a  way  that  the  sum  of  the 
payoffs  of  the  applicants  for  the  jobs  they’re  assigned  to  is  maximal.  By  doing  so  we 
capitalize  as  much  as  possible  on  the  differential  aptitudes  and  interests  of  the 
applicant  pool  to  satisfy  the  different  requirements  of  the  different  entries. 


2  For  more  detailed  descriptions  of  the  Psychometric  Model,  please  refer  to  the  bibliography.  Copies  of  the 
papers  can  be  made  available.  Contact  the  author  for  that  purpose. 
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b.  Research  objective 

For  this  research  project  it  was  intended  to  use  the  Psychometric  Model  as  a 
technology  demonstrator  for  smart  classification.  The  purpose  was  to  check  whether 
the  method  would  be  beneficial  for  use  in  different  military  recruiting  settings. 
Datasets  were  collected  and  processed  with  the  Psychometric  Model.  The 
classification  obtained  by  the  Model  was  then  compared  to  the  actual  classification 
obtained  in  the  countries  of  origin.  In  this  report  we  review  three  datasets  originating 
from  respectively  Portugal,  Spain  and  the  United  States.  A  fourth  dataset  originating 
from  the  Czech  Republic  couldn’t  be  included  since  it  didn’t  encompass  the  trade  to 
which  the  applicants  were  assigned  to  in  the  Czech  Republic.  This  prevented  us  to 
make  the  comparison  of  the  quality  obtained  by  the  Psychometric  Model  versus  the 
original  classification  method. 

4.  Results  and  discussion 

a.  The  Portuguese  data 
(1)  Description 

The  data  was  provided  by  Major  Rui  Ribeiro  and  originates  from  the  recruiting 
of  non-commissioned  officers  for  the  Portuguese  Air  Force  in  2001.  The  set 
counts  261  applicant  records.  There  were  478"  vacancies  for  a  total  of  13  trades. 
The  records  include  both  selection  data  and  the  choices  of  the  applicants  (1st,  2" 
and  3rd  choices).  The  assignment  outcome  was  also  given:  from  the  261 
applicants,  162  were  assigned  to  a  job. 


Portuguese  Air  Force  data 

#  Persons 

261 

#  Available  jobs 

478 

#  Entries 

13 

#  Assigned  persons 

162 

Table  1 


The  standards  to  assign  persons  to  jobs  were  put  in  the  Psychometric  Model.  The 
used  weights  were  unit  weights  for  the  relevant  variables.  The  choices  in  the 
dataset  were  expressed  on  an  ordinal  scale  (1st,  2nd,  3rd  choice).  Since  the 
Psychometric  Model  uses  preference  data  on  a  metric  scale  (from  99  down  to  1), 
the  applicants’  choices  were  arbitrarily  translated  to  99  (first  choice),  85  (second 
choice)  and  70  (third  choice).  Interestingly  enough,  some  applicants  were 
assigned  by  the  Portuguese  Air  Force  to  jobs  that  weren’t  one  of  their  three 
choices.  Obviously,  these  applicants  will  have  accepted  that  trade,  even  if  it  was 
not  one  of  their  first  choices.  To  allow  the  Psychometric  Model  to  assign  these 
persons  to  these  jobs  as  well,  we  considered  these  entries  as  their  fourth  choice 
and  gave  it  a  metric  value  of  60.  All  other  values  were  set  to  zero,  excluding  the 
possibility  of  assigning  an  applicant  to  these  entries. 


3  It  was  reported  that  the  number  of  vacancies  was  much  larger  than  the  number  of  applicants  because  the 
number  of  vacancies  included  all  jobs  for  the  current  year  whereas  the  applicants  do  not  represent  the 
applications  for  a  complete  year. 
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(2)  Model  1  (PsyMod  Portugal  Data  vl) 

In  this  model,  containing  the  original  data  and  the  selection  criteria  as  we 
understood  from  the  Portuguese  documentation,  something  obviously  went 
wrong.  From  the  261  applicants  in  the  dataset,  only  three  got  a  non-zero  payoff 
and  were  accordingly  assigned  to  a  job.  This  indicates  that  our  understanding  of 
the  selection  rules  was  not  correct  since  162  persons  were  assigned  to  a  job  by 
the  Portuguese  Air  Force,  obviously  meeting  the  requirements.  It  therefore  was 
decided  to  adapt  the  selection  standards  in  the  Model  to  those  that  were 
effectively  used  by  the  Portuguese.  That  was  done  in  Model  2. 

(3)  Model  2  (PsyMod  Portugal  Data  v2) 

As  said  in  previous  paragraph,  for  this  Model,  the  selection  standards  for  the 
different  trades  were  adapted.  We  went  back  to  the  original  data  and  looked  for 
each  trade,  at  the  minimum  and  maximum  value  in  the  distribution  of  scores  of 
the  persons  assigned  by  the  Portuguese  Air  Force  to  that  trade.  We  did  so  for 
each  variable  and  used  these  values  as  minimum  or  maximum  values  in  the 
Psychometric  Model.  This  means  that  the  persons  that  will  be  assigned  to  a  trade 
by  the  Psychometric  Model  have  values  for  each  variable  that  lay  above  the 
minimal  value  (or  under  the  maximal  value)  the  persons  that  were  assigned  to 
the  entry  in  Portugal  had.  One  can  easily  understand  that  these  rules  are  more 
stringent  than  the  ones  used  by  the  Portuguese  Air  Force.  The  vacancies  were 
also  modified  to  reflect  the  jobs  that  were  filled  in  Portugal  (n  =  162).  The  next 
Model  will  then  again  use  the  478  original  vacancies. 


After  running  the  Model  2,  we  obtained  following  results. 


Original  Portuguese  Data 

Psychometric  Model  #2 

#  Persons  in  dataset 

261 

261 

#  Entries 

13 

13 

#  Vacancies 

162 

162 

#  Persons  assigned 

162 

162 

Average  payoff 

477.44 

492.93  (A  NS) 

Average  rank  of  choice 

1.30 

1.30 

Table  2 


In  both  cases,  all  jobs  are  filled.  The  Psychometric  Model  yields  a  somewhat 
better  average  payoff  and  respects  the  preferences  of  the  applicants  equally  well. 

(4)  Model  3  (PsyMod  Portugal  Data  v3) 

In  the  third  model,  the  original  vacancies  were  used  again.  The  other  settings 
remained  similar  to  the  second  Model.  Following  results  were  obtained. 
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Original  Portuguese  Data 

Psychometric  Model  #3 

#  Persons  in  dataset 

261 

261 

#  Entries 

13 

13 

#  Vacancies 

478 

478 

#  Persons  assigned 

162 

197 

Average  payoff 

477.44 

467.64  (A  NS) 

Average  rank  of  choice 

1.30 

1.39 

Table  3 


Now,  the  Psychometric  Model  is  able  to  assign  more  people  in  the  dataset  to  a 
job  than  it  was  possible  in  Portugal.  23  applicants  more  could  get  a  job  (an 
increase  of  14  %).  The  price  to  pay  is  a  slight  decrease  in  quality  (average  payoff 
467  instead  of  477,  non-significant  difference)  but  remember  that  all  persons 
assigned  to  an  entry  have  selection  scores  within  the  range  of  the  persons 
accepted  for  that  entry  by  the  Portuguese  Air  Force.  It  should  be  noted  that  only 
197  persons  were  assigned  to  a  job  and  not  the  261  applicants  in  the  dataset 
since  only  197  met  the  requirements  for  at  least  one  entry. 

(5)  Conclusions  for  the  Portuguese  dataset. 

When  using  the  Psychometric  Model  on  the  Portuguese  Air  Force  data,  little 
change  can  be  observed  in  comparison  to  the  outcome  in  the  original  setting. 
When  setting  the  vacancies  to  those  that  were  filled  in  Portugal,  these  vacancies 
can  be  filled  by  the  Psychometric  Model  and  only  a  slight  (non  significant) 
increase  of  the  average  payoff  is  seen.  On  the  other  hand,  when  the  original 
vacancies  are  used,  then  the  Model  is  able  to  assign  more  persons  to  jobs  than  it 
was  possible  in  Portugal.  This  is  not  surprising  given  the  used  methodology.  In 
times  where  it  is  hard  to  achieve  recruitment  goals,  this  should  be  considered  an 
important  positive  result. 

b.  The  Spanish  data 
(1)  Description 

Capt  Psych  Jose  Puente  provided  us  with  the  Spanish  data.  It  contains  615 
records.  All  615  applicants  in  the  dataset  were  assigned  to  a  job.  There  were  49 
different  trades  but  the  applicants  were  assigned  to  only  8  of  them. 


Spanish  data 

#  Persons 

615 

#  Available  jobs 

615 

#  Entries 

49 

(8  effective) 

#  Assigned  persons 

615 

Table  4 


The  dataset  contained  the  first  two  choices  of  the  applicants.  When  checking  the 
original  data,  it  appeared  that  45  persons  were  assigned  to  jobs  not  mentioned  as 
their  first  or  second  choice.  Obviously  it  was  possible  to  assign  persons  to  other 
jobs  than  to  their  first  two  choices.  In  the  Psychometric  Model  it  was  therefore 
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decided  to  translate  the  choices  to  metric  preferences  as  follows:  first  choice  - 
99,  second  choice  =  80,  other  entries  =  1. 

(2)  Model  1 

The  Model  encompasses  615  persons  and  exactly  the  same  number  of  vacancies. 
The  vacancies  were  set  to  the  jobs  that  the  applicants  were  assigned  to  by  the 
Spanish  Military.  The  most  important  results  of  the  classification  by  the 
Psychometric  Model  in  comparison  to  the  original  classification  in  Spain  are 
shown  in  the  next  table. 


Original  Portuguese  Data 

Psychometric  Model  #1 

#  Persons  in  dataset 

615 

615 

#  Entries 

8 

8 

#  Vacancies 

615 

615 

#  Persons  assigned 

615 

615 

Average  payoff 

507,26 

513,44  (A  NS) 

#  Persons  getting 
their  1st  choice 

498 

501 

#  Persons  getting 
their  2nd  choice 

72 

88 

#  Persons  getting 
a  trade  other  than  their  1st  or 
2nd  choice 

45 

26 

Table  5 


(3)  Conclusions  for  the  Spanish  data 

The  Psychometric  Model  is  able  to  fill  all  vacancies  but  yields  only  a  slight 
increase  of  average  payoff  (non-significant)  and  a  little  more  respect  of  the 
applicants’  choices. 

c.  The  US  Air  Force  data 
(1)  Description 

This  data  was  provided  in  coordination  with  Dr  Paul  DiTullio,  Dr  Mary  Ann 
Lightfoot  and  Ted  Diaz.  Actually  two  sets  were  given:  one  composed  of  23578 
records  of  enlisted  personnel  for  the  US  Air  Force  in  1996  and  the  other  one 
containing  29246  persons  enlisted  in  2000.  For  this  report,  only  the  1996  dataset 
will  be  discussed.  146  different  trades  (entries)  were  available  and  all  persons  in 
the  dataset  were  all  assigned  to  a  job 


US  Air  Force  data  1996 

#  Persons 

23578 

#  Available  jobs 

23578 

#  Entries 

146 

#  Assigned  persons 

23578 

Table  6 

The  dataset  contained  composite  scores  that  determined  whether  or  not  an 
applicant  was  eligible  for  an  entry.  The  dataset  didn’t  contain  variables  referring 
to  the  trade  preferences  of  the  applicants.  In  the  Psychometric  Model,  it  was 
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therefore  assumed  that  the  applicants  were  equally  interested  in  all  trades.  We 
will  discuss  that  assumption  later. 

Gender  and  race  were  provided  as  categorical  data.  The  assignment  of  females 
or  persons  belonging  to  an  ethnic  minority  can  be  encouraged  in  the 
Psychometric  Model  by  using  a  multiplicative  coefficient  larger  than  one.  In  this 
research  however  this  was  not  necessary  since  in  the  dataset,  the  number  of  jobs 
equals  the  number  of  applicants.  In  such  circumstances  it  is  anticipated  that  all 
applicants  get  a  job  on  the  condition  that  the  selection  variables  are  reasonably 
unbiased  for  gender  and  ethnicity  and  that  the  standards  to  be  eligible  for  the 
trades  do  not  reject  a  too  high  proportion  of  the  applicant  pool. 

Running  the  Psychometric  Model  on  such  a  large  dataset  proved  not  to  be 
feasible  due  to  limitations  of  the  used  hardware  (1  Gigahertz  processor  and  256 
MB  RAM).  A  major  problem  is  due  to  the  fact  that  a  matrix  of  dimension  n  has 
to  be  processed,  where  n  is  the  number  of  persons  or  jobs,  whichever  is  largest. 
In  the  case  of  the  1996  dataset,  this  would  be  a  matrix  containing  556  million 
cells!  The  classification  algorithm  has  no  limitations  in  terms  of  numbers  that  it 
can  handle.  Using  a  more  powerful  computer  could  therefore  solve  the  problem. 
It  is  however  unlikely  that  in  real  circumstances,  a  batch  classification  model 
will  have  to  deal  with  such  large  numbers.  Batch  classification  indeed  supposes 
that  applicants  are  assessed  before  the  classification  takes  place.  Assessing  a 
large  pool  of  applicants  in  order  to  get  a  large  dataset  for  the  classification  will 
prove  beneficial  for  the  classification  outcome  but  it  has  a  price.  The  price  is  that 
the  applicants  who  were  assessed  first  will  probably  have  to  wait  for  a  long  time 
before  they  get  the  results  of  the  classification.  That  is  something  we  want  to 
avoid.  It  would  therefore  be  better  to  select  a  smaller  amount  of  applicants  and 
perform  the  classification  more  frequently. 

In  this  research,  we  took  subsets  of  the  original  dataset.  We  made  three  subsets 
and  took  the  first  1000,  2000  and  3000  records  of  the  original  set4.  The  available 
jobs  in  the  Psychometric  Model  were  set  to  reflect  the  jobs  that  the  applicants 
were  originally  assigned  to  in  the  US. 

(2)  Model  1  (USAF  1996  first  lk) 

The  1000  jobs  to  which  the  first  1000  persons  of  the  USAF  1996  dataset  were 
assigned  encompassed  128  different  trades.  The  Model  was  run  with  the 
selection  data  and  assignment  rules  given  by  the  US  Air  Force. 


US  Air  Force  data  1996  (first  1000) 

#  Persons 

1000 

#  Available  jobs 

1000 

#  Entries 

128 

#  Assigned  persons 

992 

Tabic  7 


4  Records  were  sorted  randomly  in  the  original  dataset. 
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Only  992  jobs  were  filled  with  the  first  run  of  the  Model.  The  8  persons  who 
didn’t  get  a  job  had  on  average  an  AFQT5 6  score  that  laid  one  and  a  half  standard 
deviation  below  the  average  of  the  1000  persons  in  the  Model.  They  didn’t  meet 
the  requirements  for  the  jobs  that  were  unfilled.  The  reason  why  these  persons 
were  not  assigned  to  a  job  is  that  their  payoff  for  the  different  trades  was  so  low, 
that  a  better  sum  of  payoffs  could  be  reached  without  them  than  when  assigning 
them  to  a  job  and  having  to  reassign  the  person  that  they  would  take  the  place 
off.  The  table  below  gives  the  payoff-values  these  persons  have  for  the  job  they 
got  from  the  USAF  assignment  and  shows  what  their  maximal  payoff  is  for  any 
trade  in  the  Psychometric  Model.  Given  that  the  average  payoff  for  each  trade  is 
set  to  500  with  a  standard  deviation  of  200,  one  can  easily  see  that  these 
applicants  were  indeed  poor  performers. 


Person-ID 

Payoff  for  Job  they  were 
assigned  to  by  the  USAF 

Maximum  Payoff  in  PsyMod 

199600001 

147 

174 

199600146 

121 

227 

199600418 

159 

159 

199600198 

352 

386 

199600234 

194 

237 

199600543 

154 

284 

199600702 

178 

364 

199600169 

170 

Table  8 


This  can  be  considered  as  a  drawback  of  the  Psychometric  Model  compared  to 
the  actual  assignment  method  used  by  the  US  Air  Force.  However,  one  has  to 
consider  that  the  setting  is  a  bit  artificial  in  the  sense  that  the  number  of  jobs 
matches  the  number  of  applicants.  In  a  more  typical  situation  where  the  number 
of  applicants  easily  exceeds  the  number  of  jobs,  this  wouldn’t  be  an  issue. 

After  having  run  the  Psychometric  Model,  we  are  in  the  possession  of  two 
datasets  that  can  be  compared.  Both  contain  the  same  persons  with  the  same 
selection  scores  and  identical  vacancies  for  which  the  same  standards  apply.  One 
originates  from  the  actual  assignment  performed  by  the  US  Air  Force  and  the 
other  one  from  the  assignment  proposed  by  the  Psychometric  Model.  In  order  to 
assess  the  quality  of  the  classification,  we  will  compare  two  things  that  are 
possible  with  these  two  datasets. 

One  of  the  traditional  ways  of  comparing  the  classification  results  is  to  look  at 
the  mean  payoffs.  While  preparing  the  Psychometric  Model,  a  payoff  value  is 
computed  for  each  applicant-job  combination.  That  payoff  is  the  expression  of 


5  AFQT:  The  Armed  Forces  Qualification  Test  (AFQT)  percentile  is  an  overall  measure  of  how  well  the 
applicant  performed  in  the  Armed  Forces  Vocational  Aptitude  Battery  (ASVAB).  This  score  is  used  to  determine 
the  initial  qualification  for  or  selection  into  the  Air  Force  of  an  individual.  It  is  not  employed  for  trade  (AFSC) 
eligibility  after  the  inidividual  meets  the  minimum  qualifying  AFQT. 

6  In  our  opinion,  this  person  doesn’t  meet  the  requirements  for  the  job  she  has  been  assigned  to  by  the  USAF. 
She  was  assigned  to  the  trade  2W131E  for  which  the  minimal  requirements  are  a  score  £61  for  the  Mechanical 
composite  or  a  score  £46  for  the  Electronics  composite.  This  person  however  had  scores  34  (Mech)  and  31 
(Elec).  That  is  why  the  Psychometric  Model  considered  the  person  as  non-eligible  for  this  trade  and  gave  her  a 
zero-payoff. 
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how  well  the  applicant  is  suited  for  the  job.  Per  entry,  the  payoff-values  are 
standardized  with  an  average  of  500  and  a  standard  deviation  of  200.  1  and  999 
are  the  lower  and  upper  limits.  By  looking  at  the  average  of  the  payoffs  that  the 
assigned  applicants  had  for  the  jobs  they  were  assigned  to,  we  get  an  idea  of  how 
well  the  person-job  match  was  realized  by  the  used  classification  method. 


Average  of  the  payoffs  that  the  assigned  applicants  had  for  the  jobs  they  were  assigned  to. 

Original  US  Air  Force  Assignment 

Psychometric  Model  Assignment 

468,79 

581,38 

Table  9 


This  is  a  quite  impressive  difference.  On  average,  the  payoff  is  more  than  half  a 
standard  deviation  higher  when  using  a  smart  classification  system.  A  word  of 
caution  is  needed  though  before  jumping  to  conclusions.  Since  the  original 
dataset  didn’t  contain  indications  concerning  the  choices  or  preferences  of  the 
applicants,  the  Psychometric  Model  couldn’t  take  these  into  account  and  had 
complete  freedom  in  assigning  an  applicant  to  the  best-suited  job.  In  reality 
however,  such  a  situation  never  occurs  and  it  is  quite  clear  that  in  the  USAF 
assignment,  the  choices  of  the  applicants  did  play  a  role  that  limited  the  degrees 
of  freedom  to  reach  an  optimal  assignment.  The  comparison  is  therefore 
probably  biased  but  it  would  be  quite  hard  to  estimate  to  what  extend.  For 
further  research,  it  is  therefore  recommended  to  include  some  measures  of  the 
applicants’  preferences  towards  the  different  trades. 

The  average  payoffs  per  entry  obtained  by  the  two  classification  methods  are 
given  as  an  enclosure.  When  checking,  one  will  see  that  for  the  125  entries  for 
which  both  methods  assigned  persons  to,  in  101  cases  the  Psychometric  Model 
yields  a  higher  average  payoff,  reflecting  a  better  recruit  quality. 

A  second  way  of  comparing  the  original  assignment  method  with  the 
Psychometric  Model  is  to  have  a  look  at  the  means  of  the  variables  that  are 
considered  relevant  for  each  trade.  We  made  the  following  table  with  that 
purpose  in  mind.  The  table  reviews  the  different  entries.  For  clarity  and  in  order 
to  avoid  small  numbers  effects,  we  limited  the  table  to  those  entries  having  10  or 
more  jobs.  The  columns  represent  from  left  to  right: 

•  The  Job-ID  used  in  the  Psychometric  Model; 

•  The  Job-name  or  AFSC  used  by  the  US  Air  Force; 

•  The  number  of  jobs  (that  is  the  number  of  observations  for  the  averages); 

•  The  criteria  that  have  to  be  met  to  be  eligible  for  the  entry.  Some  entries 
feature  two  criteria.  When  both  have  to  be  met,  the  second  criteria  is 
preceded  by  ‘&’.  When  only  one  of  both  has  to  be  met,  the  second  one  is 
preceded  by  the  word  ‘OR’.  The  criteria  include  GEN  (for  General 
composite),  ADMIN  (for  Admin,  composite),  MECH  (for  Mechanical 
composite)  and  ELEC  (for  Electronics  composite); 

•  The  minimal  acceptable  value  for  the  composite  score  given  in  the  preceding 
column; 

•  The  average  score  for  the  row  criteria  of  the  persons  assigned  by  the  US  Air 
Force  to  the  row  job; 

•  The  average  score  for  the  row  criteria  of  the  persons  assigned  by 
Psychometric  Model  to  the  row  job. 
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Job-ID 

Job-Name 

Number  of 
jobs 

Criteria 

£  Value 

Average 

USAF 

assignment 

Average 

PsyMod 

assignment 

20 

1W031 

15 

ELEC 

50 

76.07 

90.20 

&  GEN 

64 

81.87 

91.87 

29 

2A331B 

10 

ELEC 

67 

79.20 

90.20 

34 

2A333A 

18 

MECH 

51 

72.94 

84.28 

35 

2A333B 

12 

MECH 

51 

76.83 

82.00 

42 

2A531A 

12 

MECH 

51 

75.17 

78.33 

44 

2A531C 

15 

MECH 

51 

69.6 

78.00 

60 

2A635 

11 

MECH 

57 

73.18 

93.55 

61 

2A636 

12 

MECH 

45 

71.00 

91.19 

&  ELEC 

60 

76.92 

89.42 

64 

2A733 

17 

MECH 

51 

70.71 

77.18 

77 

2E633 

14 

ELEC 

46 

70.86 

67.21 

79 

2F031 

23 

GEN 

39 

61.26 

81.57 

&  MECH 

51 

69.30 

80.39 

87 

2S031 

42 

ADMIN 

45 

69.02 

87.90 

OR  GEN 

43 

43.79 

63.43 

88 

2T031 

13 

ADMIN 

40 

71.92 

79.92 

90 

2T231 

11 

ADMIN 

32 

68.45 

86.36 

&  MECH 

51 

68.73 

65.36 

95 

2W031 

44 

MECH 

61 

66.91 

60.00 

OR  ELEC 

46 

68.23 

65.41 

97 

2W131E 

10 

MECH 

61 

50.30 

61.90 

OR  ELEC 

46 

59.30 

67.20 

103 

3A031 

35 

ADMIN 

32 

66.60 

71.63 

104 

3C031 

20 

GEN 

60 

80.20 

91.10 

111 

3E131 

11 

MECH 

51 

48.55 

44.00 

OR  ELEC 

33 

51.45 

50.55 

117 

3E731 

28 

GEN 

39 

65.57 

64.30 

119 

3M031 

20 

GEN 

30 

51.20 

40.25 

120 

3P031 

105 

GEN 

35 

59.03 

53.67 

121 

3P032 

19 

GEN 

35 

65.89 

53.47 

124 

3S031 

19 

ADMIN 

45 

71.42 

85.16 

125 

4A031 

24 

GEN 

43 

61.38 

74.42 

130 

4D031 

10 

GEN 

43 

60.20 

72.20 

136 

4N031 

31 

GEN 

43 

61.55 

76.00 

145 

6F031 

10 

ADMIN 

55 

75.20 

95.20 

146 

6F032 

14 

ADMIN 

55 

76.64 

94.29 

Table  10 


In  the  37  pairs  of  averages,  the  Psychometric  Model  appears  to  yield  better 
results  in  27  pairs.  Again,  this  should  be  taken  with  caution  since  the  remark 
concerning  the  lack  of  interference  of  the  applicants’  preferences  remains  valid 
for  these  comparisons. 
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(3)  Model  2  (USAF  1996  first  2k) 

Another  model  was  run  containing  the  first  2000  applicants  of  the  original  1996 
dataset  and  the  jobs  to  which  these  applicants  were  assigned  by  the  US  Air 
Force.  Quite  similar  results  were  obtained.  From  the  2000  jobs,  the 
Psychometric  Model  could  fill  1998. 


US  Air  Force  data  1996  (first  2000) 

#  Persons 

2000 

#  Available  jobs 

2000 

#  Entries 

138 

#  Assigned  persons 

1998 

Table  11 


Comparing  the  average  payoffs: 


Average  of  the  payoffs  that  the  assigned  applicants  had  for  the  jobs  they  were  assigned  to. 

Original  US  Air  Force  Assignment 

Psychometric  Model  Assignment 

466,05 

577,52 

Table  12 


(4)  Model  3  (USAF  1996  first  3k  vl.bpm) 

The  same  procedure  was  applied  to  the  first  3000  applicants,  again  yielding  very 
similar  results. 


US  Air  Force  data  1996  (first  3000) 

#  Persons 

3000 

#  Available  jobs 

3000 

#  Entries 

140 

#  Assigned  persons 

2986 

Table  13 


Comparing  the  average  payoffs: 


Average  of  the  payoffs  that  the  assigned  applicants  had  for  the  jobs  they  were  assigned  to. 

Original  US  Air  Force  Assignment 

Psychometric  Model  Assignment 

471,12 

582,33 

Table  14 


(5)  Further  investigations 

During  the  research  process,  the  idea  grew  that  it  would  be  worthwhile  to 
simulate  a  situation  in  the  US  where  Air  Force  applicants  would  be  assigned  to 
jobs  using  a  frequently  run  batch  classification  system. 
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To  simulate  such  an  approach,  we  processed  the  1996  dataset  sliced  in  parts  of 
1000  persons.  Given  the  total  number  of  23578,  that  gives  us  23  groups  of  1000 
applicants  and  one  group  of  the  remaining  578.  Conceptually,  this  would  be 
equivalent  to  having  a  continuous  selection  process  and  a  batch  classification 
approximately  every  fortnight.  It  is  trivial  that  such  a  procedure  yields  a  less 
powerful  solution  than  processing  the  whole  group  together  but  as  mentioned 
earlier,  this  could  be  a  more  practical  scenario. 

When  comparing  the  results  at  the  payoff  level  between  the  classification 
methods  for  the  different  subgroups,  we  get  a  confirmation  of  the  results 
reported  in  the  preceding  sections.  The  next  table  shows  for  the  assigned 
applicants,  their  mean  payoff  for  the  trade  they  were  assigned  to  by  the  two 
compared  methods. 


Applicants 

USAF 

Assignment 

PsyMod 

Assignment 

Difference 
in  payoff 

#  Assigned 

Average 

Payoff 

#  Assigned 

Average 

Payoff 

K1  (#0001-0999) 

1000 

468.79 

992 

581.38 

+112.9 

K2  (#  1000-1999) 

1000 

463.31 

994 

574.24 

+110.93 

K3  (#  2000-2999) 

1000 

466.06 

997 

574.29 

+108.23 

K4  (#  3000-3999) 

1000 

475.67 

992 

580.30 

+  104.63 

K5  (#  4000-4999) 

1000 

468.82 

993 

578.63 

+  109.81 

K6  (#5000-5999) 

1000 

464.08 

996 

576.32 

+  112.24 

K7(#  6000-6999) 

1000 

471.11 

995 

580.47 

+  109.36 

K8  (#  7000-7999) 

1000 

473.90 

995 

580.58 

+  106.68 

K9  (#  8000-8999) 

1000 

466.49 

992 

583.21 

+  116.72 

K10  (#  9000-9999) 

1000 

478.05 

995 

581.27 

+103.22 

K11  (#  10000-10999) 

1000 

475.33 

991 

590.39 

+  115.06 

K12  (#  11000-11999) 

1000 

469.48 

997 

574.36 

+  104.88 

K13(#  12000-12999) 

1000 

464.61 

993 

574.77 

+  110.16 

K14(#  13000-13999) 

1000 

467.78 

997 

580.56 

+  112.78 

K15(#  14000-14999) 

1000 

470.56 

996 

574.25 

+  103.69 

K16(#  15000-15999) 

1000 

464.22 

992 

577.37 

+  113.15 

K17(#  16000-16999) 

1000 

467.23 

987 

581.25 

+  114.02 

K18  (#  17000-17999) 

1000 

479.50 

1000 

586.61 

+  107.11 

K19(#  18000-18999) 

1000 

462.37 

995 

575.30 

+  112.93 
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Applicants 

USAF 

Assignment 

PsyMod 

Assignment 

Difference 
in  payoff 

#  Assigned 

Average 

Payoff 

#  Assigned 

Average 

Payoff 

K20  (#  19000-19999) 

1000 

481.80 

995 

586.38 

+104.58 

K21  (#  20000-20999) 

1000 

477.33 

995 

580.35 

+103.02 

K22  (#  21000-21999) 

1000 

471.50 

997 

589.98 

+118.48 

K23  (#  22000-22999) 

1000 

467.46 

998 

572.44 

+104.98 

K24  (#  23000  -  23578) 

578 

475.91 

578 

575.33 

+99.42 

Total 

23578 

470.38 

23452 

579.66 

+109.28 

Table  15 


The  ‘payoffs’  shown  in  the  previous  table  are  values  that  are  internal  to  the 
classification  system.  Within  each  trade,  the  raw  payoff  values  of  the  applicants 
meeting  the  requirements  to  be  eligible  for  the  trade  are  standardized  to  a  mean 
of  500  and  a  standard  deviation  of  200.  As  such,  the  payoffs  are  not  suitable  to 
be  incorporated  in  broader  evaluation  systems  that  would  encompass  elements 
such  as  the  effect  of  recruitment  efforts  on  applicant  quality  or  the  relationship 
between  applicant  quality  and  training  costs. 

For  that  purpose,  it  is  necessary  to  go  back  to  the  original  selection  variables.  In 
the  considered  dataset,  these  are  the  composite  scores  Mechanical,  General, 
Electronics  and  Admin.  We  will  look  at  the  composites  that  are  considered 
relevant  for  the  trades.  These  are  the  composites  for  which  a  minimum  value  is 
set,  thus  defining  who  is  eligible  for  assignment  to  the  trade.  In  attachment  C,  a 
comparison  is  given  of  the  average  score  for  the  relevant  composites  yielded  by 
the  USAF  and  the  Psychometric  Model  classification  for  each  trade.  Next  table 
gives  a  summary  of  the  attachment.  Here,  we  show  the  averages  for  each 
relevant  composite  over  the  different  trades.  In  the  first  row  for  instance  we  look 
at  the  Mechanical  composite  score.  The  ‘number  of  observations’  column  refers 
to  the  number  of  persons  that  were  assigned  to  a  trade  for  which  a  minimum 
score  was  set  for  the  Mechanical  composite.  The  total  number  of  observations  is 
larger  than  the  number  of  persons  in  the  dataset  because  there  are  trades  for 
which  more  than  one  composite  score  is  considered  relevant. 


Composite 

USAF 

Assignment 

PsyMod 

Assignment 

Difference 

#  observations 

Average 

#  observations 

Average 

Mechanical 

7826 

69.19 

7826 

73.80 

+4,6 

General 

10608 

62.75 

10608 

68.82 

+6,06 

Electronics 

6050 

73.57 

5924 

77.42 

+3,85 

Admin 

3587 

69.99 

3587 

85.16 

+  15,17 

Total 

28071 

27945 

Tabl 

e  16 

This  table  illustrates  the  core  of  what  is  meant  by  the  title  of  this  research 
project.  Remember  that  the  two  compared  classification  methods  consider  the 
same  applicants  for  the  same  vacancies.  Both  methods  fill  the  vacancies  Yet, 
the  available  indicators  of  the  Military  recruit  quality  (in  this  example,  the 
relevant  composite  scores)  are  significantly  better  when  the  classification  is  done 
by  a  smart  classification  system  such  as  the  Psychometric  Model.  The  benefits  of 
using  such  a  method  should  be  estimated  by  comparing  the  costs  of 
implementing  it  versus  the  costs  of  alternative  methods  needed  to  reach  a  similar 


7  As  mentioned  earlier,  the  Psychometric  Model  does  not  fill  all  vacancies  but  only  99.5  %.  As  was  explained, 
this  is  most  likely  due  to  the  fact  that  the  number  of  applicants  equals  the  number  of  vacancies,  which  is  an 
exceptional  situation. 
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quality  increase,  such  as  increased  advertising,  recruitment  bonuses  etc.  Given 
the  marginal  costs  of  smart  classification,  there  is  little  doubt  that  using  such  a 
method  is  the  easiest  way  to  augment  recruit  quality. 

Another  issue  we  wanted  to  investigate  further  is  that  of  the  vacancies  unfilled 
by  the  Psychometric  Model.  Out  of  the  23578  vacancies,  126  are  left  unfilled. 
When  reviewing  the  different  Models  containing  1000  jobs  and  applicants  each, 
it  became  clear  that  the  unfilled  vacancies  were  not  randomly  distributed.  The 
worst  problem  for  instance,  occurred  for  Job-ID  23  (trade  2A131)  for  which  the 
Psychometric  Model  only  filled  8  out  of  the  77  vacancies.  In  the  Model  K5 
containing  the  applicants  with  ID  199604000  to  199604999,  there  were  8 
vacancies  for  Job-ID  23  and  354  persons  with  a  payoff  larger  than  zero.  Still, 
only  one  vacancy  was  filled.  If  such  a  situation  is  unacceptable  to  the 
recruitment  manager,  he  has  options  to  solve  it.  The  Psychometric  Model  indeed 
features  the  possibility  to  set  a  higher  priority  for  specific  trades.  By  doing  so, 
the  payoff  values  for  that  trade  are  multiplied  by  the  priority.  Priorities  larger 
than  one  will  increase  the  payoffs  and  that  results  in  a  higher  probability  of 
assigning  a  person  to  that  trade.  This  usually  solves  the  problem  of  unfilled 
vacancies  for  a  particular  trade  (given  that  there  are  enough  persons  meeting  all 
requirements  for  that  trade).  Yet,  this  does  not  necessarily  means  that  the  overall 
number  of  unfilled  vacancies  will  be  lower.  For  instance,  we  can  look  at  what 
happens  in  the  Model  K5  when  we  set  the  priority  for  Job-ID  23  to  2.  When  the 
Model  is  run  with  that  priority,  all  8  vacancies  for  Job-ID  23  are  filled  (and  the 
manager’s  problem  seems  to  be  solved),  but  6  out  of  the  7  persons  who  didn’t 
get  a  job  in  the  first  place  are  still  not  assigned  to  a  job8!  Now,  6  vacancies  from 
different  trades  are  not  filled.  In  summary,  the  use  of  priorities  usually  can 
influence  for  what  trades  unfilled  vacancies  occur  but  not  necessarily  lower  the 
overall  number  of  unfilled  vacancies. 

(6)  Conclusions  for  the  US  Air  Force  dataset 

The  Psychometric  Model  encountered  problems  due  to  the  size  of  the  USAF 
dataset.  These  problems  are  due  to  hardware  limitations  and  would  most  likely 
be  solved  when  using  more  powerful  computers  or  an  alternative  optimization 
algorithm.  It  is  however  not  realistic  to  anticipate  batch  classification  situations 
exceeding  one,  two  or  three  thousand  applicants  and/or  vacancies.  Therefore,  no 
attempt  was  made  to  overcome  the  hardware  limitations. 

In  the  Models  implemented  for  this  research,  the  average  payoff  yielded  by  the 
Psychometric  Model  is  markedly  (more  than  half  a  standard  deviation)  and  (of 
course)  highly  significantly  better  than  what  was  obtained  in  the  US.  Since  the 
dataset  didn’t  include  information  concerning  the  applicants’  preferences 
towards  the  different  entries,  it  was  not  possible  to  take  that  information  into 
account.  It  can  reasonably  be  assumed  that  including  the  preference  information 


Persons  not  assigned  in  Model  K5 

Not  assigned  persons  in  K5 
with  Priority  2  for  Job-23 

199604340 

199604340 

199604421 

199604421 

199604598 

199604598 

199604834 

199604834 

199604672 

199604672 

199604364 

199604364 

199604800 
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in  the  Model  would  reduce  the  average  payoff  since  it  is  unlikely  that  all 
applicants  would  agree  to  sign  up  for  the  job  they’re  best  fit  for  irrespective  of 
their  preference.  The  magnitude  of  that  effect  is  hard  to  predict  but  based  upon 
experience  with  other  datasets  it  is  most  unlikely  that  this  alone  can  explain  the 
marked  difference  in  average  payoffs  between  the  US  outcome  and  the 
classification  with  the  Psychometric  Model. 

5.  Conclusions 

Due  to  the  impressive  number  of  parameters  involved  in  classification  questions,  it  is 
quite  difficult  to  model  the  outcome  of  particular  classification  settings.  That  is,  unless 
very  elementary  and,  as  a  consequence,  much  less  powerful  classification  methods  are 
used.  It  is  therefore  necessary  to  assess  the  power  of  a  classification  method  by  testing  it 
on  a  wide  variety  of  classification  problems  and  to  compare  its  outcome  to  that  of 
alternative  methods.  That  is  what  was  done  with  this  research  project.  Based  upon  these 
particular  datasets,  we  can  conclude  that  the  Psychometric  Model  did  at  least  as  well  as 
the  different  methods  it  was  compared  to.  In  addition,  it  showed  to  be  able  to  assign  more 
people  to  jobs  without  decreasing  the  average  payoff  significantly  or  respecting  the 
applicants’  preferences  less.  In  the  case  of  the  US  Air  Force  dataset,  the  Psychometric 
Model  showed  to  be  able  to  improve  the  quality  of  the  recruited  applicants  strikingly  as  is 
highlighted  by  the  big  difference  in  average  payoff. 

Future  research  should  be  carried  out  to  increase  the  number  of  datasets  used  to 
demonstrate  the  benefits  of  smart  classification.  It  is  recommended  that  such  datasets 
include  all  eligible  applicants  and  not  only  those  who  got  a  job  in  the  original  situation 
and  that  information  concerning  their  preferences  toward  the  entries  would  be  included  as 
well. 

Classification  methodologies  deal  with  variables  and  business  rules  that  are  specific  for 
particular  countries  or  classification  settings.  Yet,  the  methodologies  can  easily  be 
implemented  in  different  countries  or  settings.  Military  recruitment  settings  are  different 
to  many  civilian  recruitment  problems  where  multiple-job  environments  are  more  the 
exception  than  the  rule.  Classification  technology  is  for  these  two  reasons  a  topic  that  is 
very  well  suited  for  international  military  collaboration.  It  is  therefore  hoped  that  the 
research  effort  made  possible  by  the  US  Office  of  Naval  Research  can  be  sustained  in  the 
future. 

6.  Addendum:  The  Sequential  Parallel  Assignment  Method,  a  valuable  alternative? 

a.  Introduction 

While  the  collaborative  research  was  conducted,  the  Belgian  Minister  of  Defense 
initiated  a  working  group  with  the  objective  of  proposing  him  a  general  concept  for 
the  selection  and  classification  of  Military  personnel.  During  the  course  of  the 
workgroup,  the  classification  issue  was  discussed  and  the  Belgian  Royal  Military 
Academy  (RMA)  proposed  an  alternative  method  for  the  Psychometric  Model.  The 
main  reason  why  the  RMA  wanted  an  alternative  is  because  they  consider  the 
Psychometric  Model  to  be  too  much  of  a  ‘black  box’  that  does  not  ensure  the  best 
applicants  to  get  their  first  choice.  The  RMA  is  clearly  not  interested  in  the  mean 
aptitude  of  the  enlisted  applicants  or  the  respect  of  the  preferences  of  the  enlisted 
group.  The  RMA  is  interested  in  the  ‘best’  applicants.  They  must  get  the  guarantee 
that  they  will  be  assigned  to  the  entry  they  seek.  The  Psychometric  Model  considers 
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all  enlisted  applicants  as  being  equally  important  to  the  Military.  A  difference  in 
philosophy  can  indeed  lead  to  a  difference  in  preferred  technique. 

b.  Method 

The  method  proposed  by  the  RMA  is  called  the  ‘Sequential  Parallel  Assignment 
Method’.  Its  starting  point  is  quite  similar  to  the  one  in  the  Psychometric  Model.  For 
each  entry,  a  payoff  is  computed  as  a  weighted  sum  of  metric  variables.  The 
Sequential  method  as  conceived  by  the  RMA  does  not  accept  the  use  of  categorical 
variables.  The  preference  of  the  applicants  for  the  different  entries  is  expressed  on  an 
ordinal  scale.  Once  all  applicants  completed  the  selection  procedure,  a  table  is  made 
for  each  different  entry.  The  tables  contain  three  fields:  the  person  identification,  the 
payoff  and  the  rank  of  their  preference  for  the  table-entry  (a  value  1  indicates  that  the 
entry  is  the  first  choice  of  the  applicant,  2  is  his/her  second  choice,  etc).  The  tables  are 
sorted  in  descending  order  of  the  payoff.  When  an  applicant  is  not  interested  at  all  in 
the  entry,  zero  replaces  his/her  payoff.  The  next  figure  shows  an  example  of  such 
tables. 


Entry  A 

Entry  B 

personjdI 

PAYOFFl 

RANK. 

personjdI 

PAYOFFr 

RANK 

841117027431 

631.620426843072 

1 

84061018758 

612.825472017944 

2 

84061018758 

61 6,249861 09046 

1 

83111614135 

606.514116992172 

2 

83111614135 

611.429062799693 

1 

82051419575 

593.782770773866 

3 

84012725329 

610.653668318389 

1 

84011112159 

593.744808692962 

2 

82101535913 

610.527902066265 

1 

82101535913 

591.792082342136 

2 

83120506956 

591.731471799733 

1 

83120506956 

588.230408677886 

2 

82051419575 

574.785455860459 

1 

84081011844 

572,532302123241 

2 

84011112159 

567.904286597929 

1 

84022414342 

559.537351902767 

2 

84081011844 

566.020873249457 

1 

83042710184 

541.124713898987 

1 

83042710184 

553.5541 3589258 

2 

84030909364 

491,168613869906 

2 

84022414342 

523.539530466325 

1 

83112622737 

335.444318952178 

2 

84030909364 

467,337602558106 

1 

84032327346 

327,136622019459 

2 

83112622737 

332,634486514898 

1 

83120110543 

0 

0 

84032327346 

319,503840874053 

1 

83090319863 

278.509631871023 

2 

83090319863 

262,395273485939 

1 

84122209922 

264.852615033528 

2 

84122209922 

256.941672919938 

1 

82082803332 

249.929181859008 

3 

82082803332 

256.068943146523 

2 

83122838520 

244.58824043448 

1 

83122838520 

243.724091845221 

2 

84052726050 

158.038787339021 

4 

84052726050 

156.905453021395 

3 

84051408137 

28.3004160702855 

3 

84051408137 

25.5153205882119 

2 

83053031182 

0 

3 

83031129770 

0 

2 

83080321341 

0 

3 

83030122752 

0 

3 

83020207570 

0 

3 

83022522704 

0 

1 

83073015261 

0 

0 

83021733737 

0 

9 

83062845901 

0 

2 
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The  Sequential  Parallel  method  then  considers  the  number  of  vacancies  per  entry.  If  n 
is  the  number  of  vacancies,  then  the  method  will  assign  persons  to  the  trade  if  they  are 
among  the  n  best  ranked  persons  for  the  entry  and  the  entry  is  their  first  choice.  Say 
for  instance  that  there  are  4  vacancies  for  Entry  A  and  4  for  Entry  B  in  our  example  in 
the  figure  above.  For  Entry  A,  the  four  best  ranked  applicants  would  be  assigned  to  the 
entry  while  for  Entry  B,  nobody  would  since  the  four  best  ranked  persons  did  not  have 
Entry  B  as  first  choice.  This  would  be  done  for  all  entries. 

Once  a  person  is  assigned  to  a  job,  his  or  her  record  is  deleted  in  the  other  tables.  In 
our  example  for  instance,  we  can  see  that  person  with  Person-ID  84061018758 
(second  ranked  in  Entry  A  and  first  in  Entry  B)  is  assigned  to  entry  A  for  this  entry  is 
his/her  first  choice  and  s/he  is  among  the  4  (because  there  are  4  vacancies)  best  ranked 
for  the  entry.  Keeping  his/her  record  in  entry  B  is  then  pointless  and  therefore  it  can  be 
removed.  When  considering  the  4  (because  there  are  also  4  vacancies  for  this  entry) 
best-ranked  persons  for  entry  B,  we  now  also  have  to  consider  the  person  who  was  the 
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5th  best  ranked  for  the  entry.  In  the  event  this  entry  is  his/her  first  choice  (which  it  isn’t 
in  our  example),  s/he  would  get  entry  B. 

This  procedure  is  continued  as  long  as  persons  can  be  assigned  to  their  first  choice. 

Then,  after  adapting  the  number  of  vacancies  per  entry  (set  to  the  original  number 
minus  the  number  of  persons  assigned  to  the  entry  in  the  run  considering  first 
choices),  the  second  choice  is  considered  in  a  similar  way.  Then  the  third  choice  is 
reviewed  and  so  on. 

In  this  method,  it  can  happen  that  an  applicant  is  assigned  to  his/her  second  choice  for 
instance  because  at  that  time  s/he  doesn’t  qualify  for  the  entry  of  his/her  first  choice. 
If,  during  the  classification  process,  due  to  the  deletion  of  the  records  of  persons  who 
were  assigned  to  an  entry  of  a  higher  choice,  this  person’s  first  choice  becomes 
available  for  him/her,  then  the  person  will  be  assigned  to  his/her  first  choice  while  the 
vacancy  of  his/her  second  choice  will  be  made  available  again  for  other  persons.  The 
graphic  below  shows  the  logic  of  the  method. 

►  For  Rank  =  1  to  #  Entries  (review  each  rank) 

- ►Run  1 

r — ►For  Entry  =  1  to  #  Entries  (review  each  entry) 

— ►  For  Person  =  0  to  #  Available  jobs  (review  first  ranked  persons) 

If  PersonRank  <=  Rank  and  Payoff  >  0  then 

If  Person  already  assigned  then  delete  assignment  and  reopen  job 
assign  Person  to  Entry 

delete  Person  for  Entries  where  PersonRank  >  PersonRank 

end  if 

- Next  Person 

- Next  Entry 

- If  there  was  an  assignment  during  this  run,  do  another  run 

' - Next  Rank  _ _ 

Graph  2 

The  method  stops  as  soon  as  there  are  no  more  vacancies  or  all  choices  have  been 
reviewed. 

c.  Comparison  of  the  two  methods 
( 1 )  Aims  of  the  two  methods 

The  Sequential  Parallel  Assignment  method  (SeqPar)  makes  sure  that  the 
applicants  with  the  highest  payoffs  get  their  preferred  entry.  The  main  concern 
of  the  supporters  of  this  method  is  that  the  ‘high  potentials’  are  satisfied  for  they 
are  the  future  leaders  of  the  organization.  To  reach  that  goal,  it  is  acceptable  to 
them  that  both  the  aptitude  and  the  preferences  of  the  other  persons  that  will  be 
enlisted  are  respected  less. 

For  the  Psychometric  Model  (PsyMod),  the  solution  of  choice  is  one  that 
maximizes  the  aptitudes  and  preferences  of  the  group  of  enlisted  persons.  For  the 
Psychometric  Model  it  is  acceptable  that  an  individual  does  not  get  the  vacancy 
for  which  s/he  is  best  qualified  or  prefers  most  if  that  allows  reaching  a  better 
solution  at  the  group  level.  The  Psychometric  Model  was  developed  as  a  flexible 
tool  allowing  the  personnel  managers  to  implement  their  accession  policy. 
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(2)  Used  parameters 

(a)  The  applicant’s  aptitude  for  the  entry 

Used  in  both  methods.  The  Sequential  Parallel  method  restricts  the  used 
information  to  metric  variables  and  does  not  allow  categorical  information 
such  as  the  kind  of  studies  applicants  took. 

(b)  The  applicant’s  preferences 

Both  methods  use  preferences  but  in  a  different  way.  First,  there  is  a 
difference  in  the  way  the  applicant  has  to  express  preferences.  For  the 
Sequential  Parallel  method,  the  preferences  are  expressed  on  an  ordinal 
scale  (1st  choice,  2nd  choice,  etc.).  For  the  Psychometric  Model,  a  metric 
scale  (from  99  down  to  1)  is  used  allowing  the  applicant  to  tell  the  Military 
how  much  s/he  likes  each  entry.  This  was  introduced  after  it  was  clear  that 
an  ordinal  scale  doesn’t  give  adequate  information  about  the  perceived 
distance  between  successive  ranks.  An  example  from  the  NCO  recruiting 
will  illustrate  this  point. 


Person  A 

Person  B 

Entry 

Preference 

Entry 

Preference 

Infantry 

99 

Navy  Radio  technician 

99 

Armor 

98 

Air  Force  Electronics 

80 

Artillery 

97 

Transportation 

80 
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In  this  example,  Person  A  tells  us  that  there  is  little  difference  between  his 
three  highest  choices.  Person  B  on  the  contrary  indicates  that  to  him,  there  is 
an  important  difference  between  his  two  choices.  He  tells  us  that  he 
wouldn’t  be  very  satisfied  if  he  was  assigned  to  his  second  choice  whereas 
for  Person  A,  it  doesn’t  really  matter  which  of  his  three  first  choices  he  gets 
(but  he  wouldn’t  be  happy  with  the  fourth).  The  Sequential  Parallel  method 
only  looks  at  the  ranks  and  therefore  has  less  information.  The  question  at 
stake  is  whether  Person  A  wouldn’t  be  more  satisfied  when  he’s  assigned  to 
his  third  choice  than  Person  B  would  be  when  assigned  to  his  second 
choice?  The  Psychometric  Model  works  with  the  metric  preferences  and 
will  consequently  give  precedence  to  the  assignment  of  Person  A  to  any  of 
his  first  three  choices  compared  to  the  assignment  of  Person  B  to  his  second 
choice. 

A  second  difference  related  to  the  applicants’  preferences,  is  the  way  in 
which  these  are  processed.  The  Sequential  Parallel  method  doesn’t  integrate 
aptitudes  and  preferences.  The  method  sorts  the  applicants  for  each  trade  in 
descending  order  of  their  aptitudes  and  assigns  them  taking  the  rank  order  of 
their  choices  into  account  as  was  explained  earlier.  PsyMod  integrates 
aptitudes  and  preferences  for  each  trade.  This  originates  from  the  generally 
accepted  idea  that  the  level  of  performance  results  from  the  product  of 
aptitude  and  motivation.  The  integrated  value  of  aptitude  and  preference 
specifies  the  utility  of  assigning  a  person  to  a  particular  job.  That  is  the 
value  that  is  used  by  the  Psychometric  Model  in  the  search  of  an  optimal 
solution  for  the  classification  problem. 

A  third  facet  of  the  topic  is  that  of  the  importance  that  is  given  to  the  respect 
of  the  applicants’  preferences.  The  role  that  is  played  by  the  preferences  in 
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the  Sequential  Parallel  method  is  fixed.  It  is  known  that  there  is  an 
interaction  between  the  respect  of  the  preferences  and  the  respect  of 
aptitudes.  If  the  recruitment  manager  wants  to  maximize  aptitudes,  there  is 
only  little  room  to  respect  the  applicant’s  choices.  Conversely,  if  we  want  to 
satisfy  the  applicants,  this  will  most  probably  happen  at  the  expense  of 
aptitude.  This  is  illustrated  by  the  next  graph. 


Quality  versus  Preferences 
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Preference  weight  in  Psychometric  Model 
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This  graph  shows  for  the  same  recruitment,  the  average  aptitude  (left 
ordinate)  and  the  average  preference  (right  ordinate)  for  different  levels  of 
preference  weights  in  the  Psychometric  Model.  In  addition,  the  graph  shows 
the  fixed  level  of  average  payoff  and  average  preference  yielded  by  the 
Sequential  Parallel  method  (horizontal  lines). 

The  graph  clearly  shows  the  interaction  between  aptitude  and  preference 
levels.  If  one  wants  to  maximize  the  aptitude,  this  will  happen  at  the 
expense  of  the  respect  of  preferences  and  vice  versa.  The  Sequential  Parallel 
method  doesn’t  allow  varying  the  importance  given  to  the  preferences. 

Based  upon  empirical  evidence,  a  usual  value  given  to  the  preference  weight 
is  0.6.  When  this  value  is  set  for  the  considered  recruitment,  you  can  see  that 
the  Psychometric  Model  performs  better  both  for  the  respect  of  preferences 
and  for  yielding  higher  aptitudes. 

(c)  The  importance  for  the  organization  of  filling  vacancies 

The  Sequential  Parallel  method  offers  no  possibility  to  emphasize  the 
importance  of  filling  a  particular  trade.  The  psychometric  Model  does.  If 
there  only  is  a  very  limited  number  of  applicants  qualifying  for  a  more 
difficult  trade,  and  these  applicants  also  qualify  well  for  other  trades  that 
they  prefer,  the  risk  is  real  that  the  Sequential  Parallel  method  will  assign 
them  to  their  preferred  trade  and  find  no  suitable  applicants  for  the  more 
difficult  trade.  If  this  happens  in  the  Psychometric  Model,  the  user  will  have 
the  possibility  to  give  a  priority  to  the  difficult  trade.  This  will  usually  result 
in  the  assignment  of  sufficient  numbers  of  qualified  applicants  to  that  trade. 


23 


(3)  The  classification  algorithm 

The  Sequential  Parallel  method  follows  a  rudimentary  logic.  The  method  leads 
to  a  quasi-mechanical  solution  of  the  classification  problem  and  doesn’t  include 
any  flexibility.  The  Psychometric  Model  on  the  contrary  has  an  impressive 
number  of  degrees  of  freedom  when  optimizing  the  payoff  matrix.  This 
significantly  increases  its  flexibility  to  reach  good  solutions. 

(4)  Performance 

To  evaluate  the  performance  level  of  the  two  methods,  it  proves  necessary  to 
apply  the  methods  to  the  same  datasets.  For  that  purpose,  we  used  the  datasets  of 
the  NCO  recruitments  for  the  Belgian  Defense.  We  considered  the  recruitments 
of  1999,  2000  and  2001.  Each  year,  there  are  four  distinct  recruitments.  Two  are 
for  the  French  speaking  applicants  (marked  with  the  letter  F)  and  two  for  the 
Flemish  applicants  (marked  with  N).  For  each  language  group,  there  are  two 
possibilities:  a  recruitment  for  which  a  high  school  certificate  is  required  (called 
Level  2)  and  another  one  for  younger  applicants  (called  Level  3+).  These 
applicants  will  have  to  take  the  last  two  year  of  high  school  in  a  militant  school. 
The  12  recruitments  in  this  study  had  3167  applicants  for  983  vacancies' . 

It  has  to  be  said  that  this  approach  is  rather  irrelevant  for  the  supporters  of  the 
Sequential  Parallel  method.  To  them,  the  mean  values  of  payoffs  and  preferences 
of  the  group  that  was  assigned  to  a  job  is  not  important  as  long  as  the  best  ranked 
applicants  get  their  preferred  job.  We  do  not  agree  with  this  point  of  view  and 
present  the  results  so  that  the  reader  can  decide  for  him  or  herself  whether  the 
following  comparisons  are  meaningful. 

(a)  The  aptitude  of  the  applicants  who  are  assigned  to  a  job. 

Next  graph  shows  the  mean  aptitude  of  the  applicants  who  received  a  job. 
Three  conditions  are  given  for  the  Psychometric  Model:  a  maximal  weight 
for  the  preferences  of  the  applicants  (Pref  1,0),  the  preference  weight  that 
actually  was  used  for  these  recruitments  (Pref  0,6)  and  no  weight  for  the 
preferences  at  all  (Pref  0,0).  For  clarity,  the  mean  aptitude  yielded  by  the 
Sequential  Parallel  method  is  set  to  100  for  the  different  recruitments  as  a 
reference  point.  The  mean  aptitudes  yielded  by  the  Psychometric  Model  are 
given  on  the  same  scale. 


9  As  will  be  described  further  in  the  section  dealing  with  the  fill  rates,  the  Psychometric  Model  leads  to  fewer  unfilled  vacancies 
than  the  Sequential  Parallel  method.  Of  course,  this  has  a  price.  In  the  search  for  filling  more  vacancies,  progressively  less 
qualified  applicants  or  applicants  with  less  preference  for  the  job  are  assigned  to  jobs.  In  order  to  allow  a  fair  comparison  in  the 
data  presented  in  graphs  4  and  6,  the  vacancies  that  couldn't  be  filled  by  the  Sequential  Parallel  Method,  were  removed  from 
the  data. 
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Mean  payoff  for  assigned  applicants 

Psychometric  Model:  Weight  choices  Max  (1),  Min  (0)  and  Usual  (0.6) 
Sequential  Parallel  Method  (Set  to  1 00  as  reference) 


Recruitment  sessions 

Graph  4 

The  graph  shows  that  in  most  cases,  the  average  aptitude  is  higher  when  the 
assignment  is  performed  with  the  Psychometric  Model.  Comparisons  in 
which  the  Sequential  Parallel  method  can  yield  better  results  are  these  where 
the  Psychometric  Model  incorporates  a  (very)  high  importance  of  the 
preferences.  Graph  3  illustrated  the  reason  for  this. 

Next  graph  makes  the  transition  between  the  evaluation  of  aptitudes  and 
preferences.  The  scatter  plot  shows  the  aptitude  of  assigned  applicants  for  a 
particular  recruitment.  The  abscissa  gives  the  rank  order  of  the  choice 
expressed  by  the  applicant  for  the  entry  s/he  was  assigned  to.  Above  the 
abscissa  scale  value  1  for  example,  the  icons  represent  the  aptitudes  of  the 
applicants  who  were  assigned  to  the  entry  that  was  their  first  choice  (blue 
triangles  for  the  Psychometric  Model  and  red  circles  for  the  Sequential 
Parallel  method).  For  the  Psychometric  Model,  the  weight  that  is  given  to 
the  preferences  of  the  applicants  is  the  weight  that  effectively  was  used  in 
that  recruitment. 
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Payoffs  of  applicants  assigned  to  the  entry  of  their  n-th  choice 

Psychometric  Model:  weight  for  preferences:  0  6 
Sequential  Parallel  method:  fixed  rule 
Recruitment  Flemish  NCO  Level  2, 2000 


Graph  5 


PSYMOD 

SEQPAR 


This  graph  is  interesting  because  it  illustrates  the  fact  that,  despite  what  it 
claims,  the  Sequential  Parallel  method  doesn’t  assign  the  applicants  with  the 
highest  payoffs  to  their  preferred  entry  in  a  much  better  way  than  the 
Psychometric  Model  does. 


(b)  Respecting  the  applicants’  preferences 


The  next  graph  compares  the  average  preference  the  applicants  expressed 
for  the  trade  they  were  assigned  to.  As  explained  earlier,  with  the 
Psychometric  Model,  the  user  can  vary  the  importance  given  to  the 
expressed  preference.  The  same  three  conditions  are  shown  as  in  graph  4. 


Mean  preferences  for  assigned  applicants 


Psychometric  Model:  preference  weight  Max  (1),  Min  (0)  and  Usual  (0.6) 
Sequential  Parallel  Method:  fixed  rule 


Recruitment  sessions 


Graph  6 
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The  average  preference  obtained  by  the  Sequential  Parallel  method  lays 
between  the  boundaries  that  can  be  reached  by  varying  the  weight  given  to 
the  preferences  in  the  Psychometric  Model  When  the  weight  is  set  to  0.6  to 
balance  the  importance  of  aptitude  versus  preference,  the  Psychometric 
Model  yields  better  results. 

(c)  The  fill  rate 

In  the  twelve  analyzed  recruiting  sessions,  nor  the  Psychometric  Model  nor 
the  Sequential  Parallel  method  succeeded  in  filling  all  vacancies.  This  was 
essentially  due  to  unfavorable  selection  ratios.  Some  differences  were 
observed  and  are  given  in  the  table  below. 


Number  of  unfilled  vacancies 

Recruitment  session 

Sequential  Parallel  method 

Psychometric  Model 

NCO  Level  2  F  2000 

11 

10 

NCO  Level  2  N  2000 

2 

2 

NCO  Level  2  F  2001 

45 

30 

NCO  Level  2  N  2001 

26 

2 

Total 

84 

44 

Table  21 


It  isn’t  surprising  that  the  Sequential  Parallel  method  isn’t  able  to  fill  as 
many  vacancies  as  the  Psychometric  Model  does,  given  their  respective 
methodologies.  To  the  supporters  of  the  Sequential  Parallel  method,  the  fill 
rate  isn’t  even  of  major  concern.  The  Psychometric  Model  maximizes  the 
sum  of  payoffs  of  assigned  applicants.  In  usual  circumstances,  this  will  lead 
the  algorithm  to  try  to  assign  as  many  applicants  as  possible,  thus  reducing 
the  number  of  unfilled  vacancies.  Personnel  managers  usually  open 
vacancies  because  the  organization  needs  new  personnel.  Not  filling  these 
vacancies  is  therefore  negative  for  the  organization  and  should  be  avoided. 
A  classification  method  that  does  not  allow  incorporating  the  recruitment 
priorities  of  the  organization  nor  tries  to  maximize  the  number  of  assigned 
persons  is  therefore  less  recommendable.  For  the  sake  of  clarity,  it  is 
important  to  mention  that  the  Psychometric  Model  results  were  obtained 
without  the  use  of  priorities.  In  some  cases,  the  use  of  priorities  allows  to 
further  reduce  the  number  of  unfilled  vacancies. 
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ATTACHMENT  A:  COMPARISON  OF  AVERAGE  PAYOFFS  PER  TRADE 


Comparison  of  results  from  the  original  USAF  classification  and  the  ‘Psychometric  Model’  classification 

Recruitment  session:  PsyMod  USAF  1996  first  Ik  vl 

Number  of  entries:  146 

Number  of  vacancies:  1 000 

Number  of  persons  in  model:  1000 

Mean  preference  weight  coefficient  for  Psychometric  Model:  0,00 

Review  Entry  '1':  1A231  G  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  502,81  701,02 

Review  Entry  '3':  1 A431  G  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  772,82  680,06 

Review  Entry  *4':  1 A531  E  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  802,59  813,95 

Review  Entry  '5‘:  1 C031  A  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  554,96  674,57 

Review  Entry  '6':  1 C032  A  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  457,49  676,78 

Review  Entry  T:  1  Cl  31  G  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  319,36  680,06 

Review  Entry  ‘9’:  1C331  G  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  316,33  679,39 

Review  Entry  '1 0‘:  1 C431  G  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  407,10  679,39 

Review  Entry  '1 V:  1 C531  G  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  370,89  680,06 

Review  Entry  '1 2: 1 N031  G  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  461 ,01  706,42 

Review  Entry  *1 3‘:  1 N1 31  G  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  463,78  850,21 

Review  Entry  '1 6*:  1 N431  G  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  498,72  713,97 

Review  Entry  '17:1  N531  G  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  528,43  885,82 

Review  Entry  '18':  1 N631  G  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  355,22  713,97 

Review  Entry  '19':  1T131  G  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  533,31  251,81 

Review  Entry  ‘20*:  1 W031  E  Number  of  jobs  for  this  entry:  15 
USAF  PsyMod 

Mean  payoff:  524,56  767,73 


Review  Entry  '21':  2A031A  E  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  41 6,56  840,44 

Review  Entry  '221:  2A031 B  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  461 ,97  870,72 

Review  Entry  ‘23':  2A1 31  E  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  598,03 

Review  Entry  '25‘:  2A133  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  848,01  802,59 

Review  Entry  '2 7:  2A137  E  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  496,03  779,89 

Review  Entry  '28*:  2A331 A  E  Number  of  jobs  for  this  entry:  8 
USAF  PsyMod 

Mean  payoff:  391 ,01  677,70 

Review  Entry  *29':  2A331 B  E  Number  of  jobs  for  this  entry:  10 
USAF  PsyMod 

Mean  payoff:  489,22  739,01 

Review  Entry  ’30':  2A331C  E  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  525,55  639,10 

Review  Entry  *31':  2A332A  E  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  607,30  639,10 

Review  Entry  '32* :  2A332B  E  Number  of  jobs  for  this  entry:  8 
USAF  PsyMod 

Mean  payoff:  632,28  694,73 

Review  Entry  ‘33*:  2A332C  E  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  405,20  745,82 

Review  Entry  ’34‘:  2A333A  M  Number  of  jobs  for  this  entry:  1 8 
USAF  PsyMod 

Mean  payoff:  517,20  693,83 

Review  Entry  '35':  2A333B  M  Number  of  jobs  for  this  entry:  1 2 
USAF  PsyMod 

Mean  payoff:  577,81  658,33 

Review  Entry  ’36‘:  2A333C  M  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  595,99  705,09 

Review  Entry  ’37:  2A333E  M  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  388,20  544,04 

Review  Entry  ‘38’:  2A333H  M  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  198,58  705,09 

Review  Entry  '39':  2A431  E  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  457,43  689,05 

Review  Entry  '40':  2A432  E  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  340,86  704,19 

Review  Entry  *41':  2A433  E  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  666,35  723,12 


Review  Entry  '42:  2A531A  M  Number  of  jobs  for  this  entry:  12 
USAF  PsyMod 

Mean  payoff:  551,84  601,19 

Review  Entry  '43':  2A531 B  M  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  331 ,05  595,99 

Review  Entry  ’44* :  2A531 C  M  Number  of  jobs  for  this  entry:  1 5 
USAF  PsyMod 

Mean  payoff:  465,08  595,99 

Review  Entry  ‘45’:  2A531 D  M  Number  of  jobs  for  this  entry:  7 
USAF  PsyMod 

Mean  payoff:  500,26  636,07 

Review  Entry  ’46':  2A531 E  M  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  468,72  619,37 

Review  Entry  '47:  2A531 F  M  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  51 1 ,84  655,22 

Review  Entry  ’48':  2A531G  M  Number  of  jobs  for  this  entry:  7 
USAF  PsyMod 

Mean  payoff:  498,03  618,26 

Review  Entry  '49*:  2A531 H  M  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  558,59  543,01 

Review  Entry  '5V:  2A533A  E  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  496,03  620,93 

Review  Entry  *52:  2A533B  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  439,26  916,13 

Review  Entry  '54':  2A631 B  M  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  520,39  81 1 ,49 

Review  Entry  ’55':  2A631 D  M  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  41 5,75  340,39 

Review  Entry  '56‘:  2A631 E  M  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  343,81  374,64 

Review  Entry  ’57:  2A632  E  Number  of  jobs  for  this  entry:  8 
USAF  PsyMod 

Mean  payoff:  404,78  592,29 

Review  Entry  '58':  2A633  M  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  490,81  845,80 

Review  Entry  ‘59':  2A634  M  Number  of  jobs  for  this  entry:  9 
USAF  PsyMod 

Mean  payoff:  464,39  559,63 

Review  Entry  ’60':  2A635  M  Number  of  jobs  for  this  entry:  1 1 
USAF  PsyMod 

Mean  payoff:  470,37  831,82 

Review  Entry  *61  *:  2A636  E  Number  of  jobs  for  this  entry:  12 
USAF  PsyMod 

Mean  payoff:  491 ,67  797,01 

Review  Entry  *62:  2A731  M  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  751,84  471,31 


Review  Entry  ’63':  2A732  G  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  372,48  603,79 

Review  Entry  '64':  2A733  M  Number  of  jobs  for  this  entry:  1 7 
USAF  PsyMod 

Mean  payoff:  482,32  583,16 

Review  Entry  '65‘:  2A734  M  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  271,88  323,26 

Review  Entry  '66*:  2E031  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  552,80  643,64 

Review  Entry  '67:  2E131  E  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  484,68  711,76 

Review  Entry  '69*:  2E133  E  Number  of  jobs  for  this  entry:  7 
USAF  PsyMod 

Mean  payoff:  452,24  796,11 

Review  Entry  71’:  2E231  E  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  545,24  779,89 

Review  Entry  72:  2E231 B  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  643,64 

Review  Entry  73’:  2E331  E  Number  of  jobs  for  this  entry:  9 
USAF  PsyMod 

Mean  payoff:  530,10  714,28 

Review  Entry  74’:  2E431  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  575,51 

Review  Entry  75*.  2E631  M  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  299,88  475,21 

Review  Entry  76*:  2E632  M  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  367,42  507,68 

Review  Entry  77:  2E633  E  Number  of  jobs  for  this  entry:  14 
USAF  PsyMod 

Mean  payoff:  532,32  476,52 

Review  Entry  78':  2E831  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  212,18  689,05 

Review  Entry  79':  2F031  G  Number  of  jobs  for  this  entry:  23 
USAF  PsyMod 

Mean  payoff:  403,33  669,25 

Review  Entry  '80':  2M031A  E  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  637,96  813,95 


Review  Entry  *8V:  2M031 B  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  575,51  348,43 

Review  Entry  '82:  2M032A  M  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  459,63  673,92 

Review  Entry  '84':  2P031  E  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  484,68  757,18 


Review  Entry  '85':  2R031  G  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  458,49  690,37 

Review  Entry  '86':  2R1 31  G  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  273,19  610,34 

Review  Entry  ’87:  2S031  G  Number  of  jobs  for  this  entry:  42 
USAF  PsyMod 

Mean  payoff:  350,14  615,61 

Review  Entry  '88':  2T031  A  Number  of  jobs  for  this  entry:  13 
USAF  PsyMod 

Mean  payoff:  500,75  600,44 

Review  Entry  ‘89':  2T131  M  Number  of  jobs  for  this  entry:  7 
USAF  PsyMod 

Mean  payoff:  285,58  352,13 

Review  Entry  '90':  2T231  A  Number  of  jobs  for  this  entry:  1 1 
USAF  PsyMod 

Mean  payoff:  457,13  593,96 

Review  Entry  '91':  2T331  M  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  424,56  590,80 

Review  Entry  '94':  2T431  M  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  572,62  584,31 

Review  Entry  ’95’:  2W031  E  Number  of  jobs  for  this  entry:  44 
USAF  PsyMod 

Mean  payoff:  527,35  468,99 

Review  Entry  ’96':  2W131C  E  Number  of  jobs  for  this  entry:  9 
USAF  PsyMod 

Mean  payoff:  496,79  456,94 

Review  Entry  '97:  2W131E  E  Number  of  jobs  for  this  entry:  10 
USAF  PsyMod 

Mean  payoff:  360,64  493,60 

Review  Entry  *98’:  2W131 F  E  Number  of  jobs  for  this  entry:  9 
USAF  PsyMod 

Mean  payoff:  565,23  449,33 

Review  Entry  '99':  2W131 K  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  874,27  336,85 

Review  Entry  ‘100’:  2W131 L  E  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  579,78  61 1 ,36 

Review  Entry  ’102‘:  2W231  M  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  638,10  901,27 

Review  Entry  '1 03':  3A031  A  Number  of  jobs  for  this  entry:  35 
USAF  PsyMod 

Mean  payoff:  461,54  518,79 

Review  Entry  ’104':  3C031  G  Number  of  jobs  for  this  entry:  20 
USAF  PsyMod 

Mean  payoff:  556,57  762,65 

Review  Entry  '1 05‘:  3C032  G  Number  of  jobs  for  this  entry:  8 
USAF  PsyMod 

Mean  payoff:  718,22  770,22 

Review  Entry  '106*.  3C131  A  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  317,95  670,14 


Review  Entry  '107:  3C231  E  Number  of  jobs  for  this  entry:  7 
USAF  PsyMod 

Mean  payoff:  487,92  685,81 

Review  Entry  *1 09':  3E031  E  Number  of  jobs  for  this  entry:  3 

USAF  PsyMod 

Mean  payoff:  696,08  331 ,73 

Review  Entry  '1 1 0':  3E032  E  Number  of  jobs  for  this  entry:  5 

USAF  PsyMod 

Mean  payoff:  391 ,33  698,54 

Review  Entry  ’111':  3E131  E  Number  of  jobs  for  this  entry:  1 1 

USAF  PsyMod 

Mean  payoff:  359,89  330,46 

Review  Entry  '1 12*:  3E231  M  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  431 ,73  326,69 

Review  Entry  *1 1 3*:  3E331  M  Number  of  jobs  for  this  entry:  8 
USAF  PsyMod 

Mean  payoff:  502,48  467,42 

Review  Entry  '114':  3E431  M  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  414,17  499,89 

Review  Entry  '1 1 5':  3E432  M  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  258,32  512,87 

Review  Entry  '117:  3E731  G  Number  of  jobs  for  this  entry:  28 
USAF  PsyMod 

Mean  payoff:  507,59  499,71 

Review  Entry  '1 1 8':  3E831  G  Number  of  jobs  for  this  entry:  4 
USAF  PsyMod 

Mean  payoff:  650,95  882,94 

Review  Entry  '119':  3M031  G  Number  of  jobs  for  this  entry:  20 
USAF  PsyMod 

Mean  payoff:  358,61  231 ,49 

Review  Entry  '120':  3P031  G  Number  of  jobs  for  this  entry:  105 
USAF  PsyMod 

Mean  payoff:  439,63  375,70 

Review  Entry  '121':  3P032  G  Number  of  jobs  for  this  entry:  19 
USAF  PsyMod 

Mean  payoff:  521 ,49  373,40 

Review  Entry  '1 22':  3P032A  G  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  594,28  367,76 

Review  Entry  '1 23':  3P1 31  M  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  318,99  861,80 

Review  Entry  '124':  3S031  A  Number  of  jobs  for  this  entry:  1 9 
USAF  PsyMod 

Mean  payoff:  477,48  658,95 

Review  Entry  '1 25’:  4A031  G  Number  of  jobs  for  this  entry:  24 
USAF  PsyMod 

Mean  payoff:  425,40  596,15 

Review  Entry  '126':  4A131  G  Number  of  jobs  for  this  entry:  9 
USAF  PsyMod 

Mean  payoff:  365,20  602,33 

Review  Entry  '1 27:  4A231  E  Number  of  jobs  for  this  entry:  4 

USAF  PsyMod 

Mean  payoff:  740,15  620,93 


Review  Entry  '1 28’:  4B031  G  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  376,84  702,66 

Review  Entry  '1 29*:  4C031  G  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  332,25  710,98 

Review  Entry  '130':  4D031  G  Number  of  jobs  for  this  entry:  10 
USAF  PsyMod 

Mean  payoff:  410,01  567,13 

Review  Entry  '13V:  4E031  G  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  800,18  603,79 

Review  Entry  *1 32:  4F031  G  Number  of  jobs  for  this  entry:  5 
USAF  PsyMod 

Mean  payoff:  462,38  574,98 

Review  Entry  '1 33':  4H031  G  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  302,65  616,88 

Review  Entry  '1 34’:  4J032  G  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  553,72  693,36 

Review  Entry  ’1 36’:  4N031  G  Number  of  jobs  for  this  entry:  31 
USAF  PsyMod 

Mean  payoff:  427,67  572,1 1 

Review  Entry  '137:  4N131  G  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  464,13  608,15 

Review  Entry  *1 38':  4P031  G  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  564,51  643,07 

Review  Entry  *1 39':  4R031  G  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  538,32  643,07 

Review  Entry  '1 40':  4T031  G  Number  of  jobs  for  this  entry:  1 
USAF  PsyMod 

Mean  payoff:  893,34  713,97 

Review  Entry  '142:  4V031  G  Number  of  jobs  for  this  entry:  2 
USAF  PsyMod 

Mean  payoff:  216,31  710,98 

Review  Entry  '1 43’:  4Y031  G  Number  of  jobs  for  this  entry:  6 
USAF  PsyMod 

Mean  payoff:  409,57  643,07 

Review  Entry  '1 44’:  6C031  G  Number  of  jobs  for  this  entry:  3 
USAF  PsyMod 

Mean  payoff:  547,24  876,42 

Review  Entry  '1 45':  6F031  A  Number  of  jobs  for  this  entry:  1 0 
USAF  PsyMod 

Mean  payoff:  450,92  791 ,22 

Review  Entry  '1 46’:  6F032  A  Number  of  jobs  for  this  entry:  1 4 
USAF  PsyMod 

Mean  payoff:  491,95  776,17 


Summary  for  all  entries 

Number  of  entries  with  more  than  0  jobs:  1 28 

Number  of  jobs  for  this  model:  1 000 


USAF  PsyMod 

Total  assigned:  1000  992 

Shortfall:  0  8 

Mean  payoff:  468,79  581,38 
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ATTACHMENT  B:  AVERAGE  COMPOSITE  SCORES  PER  ENTRY 

ORIGINAL  USAF  1996  ASSIGNMENT 


Job-ID  (AFSC) 

#  Assiqned 

Mechanical 

Admin 

General 

Electronics 

1  (1A231) 

112 

61,06 

71,65 

73,23 

70,84 

2  (1 A331) 

51 

67,58 

78,94 

80,01 

81,62 

3  (1 A431) 

57 

61,35 

73,82 

73,98 

71,64 

4  (1 A531) 

27 

72,77 

80,88 

85,62 

85,33 

5  (1C031) 

50 

35,11 

70,12 

47,26 

49,68 

6  (1C032) 

195 

35,27 

69,96 

48,08 

48,83 

7  (1  Cl 31 ) 

63 

56,68 

82,82 

74,88 

70,38 

8  (1C231) 

1 

65 

74 

46 

42 

9  (1C331) 

218 

44,47 

69,78 

62,54 

59,59 

10  (1C431) 

48 

75,7 

70,66 

70,81 

73,04 

11  (1C531) 

52 

43,3 

73,21 

70,59 

65,21 

12  (1 N031) 

105 

57,11 

79,59 

76,4 

72,46 

13  (1N131) 

87 

56,51 

78,26 

78,09 

73,21 

14  (1N332A) 

11 

63 

75,63 

88,63 

84,72 

15  (1N334G) 

1 

19 

94 

74 

65 

16  (1N431) 

49 

60,34 

75,75 

74,4 

70,73 

17  (1 N531) 

45 

70,31 

75,91 

82,88 

79,4 

18  (1N631) 

24 

64,2 

78,54 

76,16 

73,58 

19  (1T131) 

139 

45,76 

62,27 

55,1 

55,25 

20  (1W031) 

290 

57,22 

76,68 

77,49 

74,63 

21  (2A031A) 

66 

65,8 

67,65 

73,31 

78,31 

22  (2A031B) 

72 

68,58 

77,44 

76,7 

78,93 

23  (2A131) 

77 

73,94 

72,85 

77,71 

83,87 

24  (2A132) 

3 

88 

74,33 

83,33 

85,33 

25  (2A133) 

16 

75,12 

79,5 

80,56 

81,25 

26  (2A134) 

7 

63,42 

74 

71,85 

80,14 

27  (2A137) 

91 

69,25 

70,57 

76,05 

79,56 

28  (2A331A) 

114 

71,3 

72,22 

73,57 

79,27 

29  (2A331 B) 

156 

70,71 

76,62 

76,2 

80,42 

30  (2A331C) 

126 

65,92 

76,38 

73,38 

77,58 

31  (2A332A) 

162 

70,1 

71,27 

74,79 

79,91 

32  (2A332B) 

100 

72,03 

72,59 

73,34 

79,55 

33  (2A332C) 

81 

72,16 

70,48 

73,76 

78,75 

34  (2A333A) 

354 

73,74 

63,58 

64,78 

68,82 

35  (2A333B) 

317 

72,35 

67,36 

64,14 

68,49 

36  (2A333C) 

17 

79 

61,58 

66,64 

72,23 

37  (2A333E) 

80 

70,55 

64,31 

61,63 

68,37 

38  (2A333H) 

32 

71,93 

63,06 

64,25 

71,21 

39  (2A431) 

121 

70,34 

75,43 

77,71 

80,88 

40  (2A432) 

126 

68,42 

75,11 

74,53 

79,47 

41  (2A433) 

81 

69,3 

74,53 

78,69 

82,32 

42  (2A531A) 

209 

71,12 

63,66 

63,7 

67,42 

43  (2A531B) 

80 

71,5 

62,48 

60,37 

64,33 

44  (2A531C) 

306 

72,94 

63,81 

63,1 

67,27 

45  (2A531 D) 

101 

73,01 

64,97 

62,64 

65,95 

46  (2A531 E) 

121 

71,76 

67,66 

65 

68,7 

47  (2A531F) 

83 

72,28 

62,69 

59,38 

65,72 

48  (2A531G) 

166 

70,42 

62,93 

62,82 

67,23 

49  (2A531H) 

116 

71,52 

60,64 

60,75 

66,12 

50  (2A532) 

17 

79,29 

65,11 

65 

70,88 

51  (2A533A) 

42 

68,66 

71,97 

74,11 

79,04 

52  (2A533B) 

45 

70 

74,66 

80,73 

80,68 

53  (2A533C) 

18 

67,16 

67,94 

71,55 

77,05 

54  (2A631B) 

85 

72,61 

59,55 

61,57 

66,75 

55  (2A631D) 

96 

62,2 

61,8 

57,16 

59,09 

56  (2A631 E) 

89 

64,32 

59,47 

56,31 

61,12 

57  (2A632) 

194 

72,73 

65,36 

63,98 

68,62 

58  (2A633) 

92 

70,76 

61,95 

59,77 

63,44 

59  (2A634) 

218 

67,16 

61,79 

58,93 

62 

60  (2A635) 

294 

72,96 

65,18 

62,5 

67,29 

61  (2A636) 

228 

70,12 

68,36 

70,83 

75,97 

62  (2A731) 

37 

74,62 

63,29 

58,16 

65,7 

63  (2A732) 

105 

40,86 

67,22 

59,02 

57,2 

64  (2A733) 

364 

69,5 

61,77 

61,16 

65,35 

65  (2A734) 

71 

57,45 

60,39 

51,42 

54,97 
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Job-1  D  (AFSC) 

#  Assigned 

Mechanical 

Admin 

General 

Electronics  1 

66  (2E031) 

38 

63,57 

76,26 

74,26 

78,47 

67 (2E131) 

128 

72,25 

75,57 

78,65 

82,6 

68  (2E132) 

20 

73,8 

76,65 

77,9 

82,45 

69  (2E133) 

97 

70,42 

74,54 

75,69 

80,55 

70  (2E134) 

25 

60,04 

76,48 

71,8 

79,2 

71  (2E231) 

81 

68,28 

73,29 

76,3 

79,3 

72  (2E231B) 

18 

76,27 

77,5 

87,11 

86 

73 (2E331) 

225 

68,38 

72,15 

74,59 

79,91 

74  (2E431) 

16 

72,06 

71,87 

77,68 

80,62 

75  (2E631) 

86 

67,12 

61,13 

55,5 

59,15 

76  (2E632) 

125 

69,72 

64,38 

60,32 

63,25 

77  (2E633) 

377 

57,01 

66,15 

63,84 

67,76 

78  (2E831) 

29 

71,41 

76,89 

77,55 

81,48 

79 (2F031) 

456 

68,65 

63,6 

60,7 

63,5 

80  (2M031  A) 

108 

71,19 

73,33 

76,79 

80,82 

81  (2M031 B) 

74 

71,58 

74,68 

76,71 

81,41 

82  (2M032A) 

121 

70,66 

66,18 

63,57 

67,38 

83  (2M033A) 

62 

55,16 

67,69 

64,74 

69,72 

84  (2P031) 

67 

71,7 

74,94 

77,1 

81,37 

85  (2R031) 

45 

49,33 

76,46 

69,93 

65,59 

86  (2R131) 

93 

39,09 

65,56 

56,91 

53,27 

87  (2S031) 

932 

36,87 

68,47 

47,82 

49,41 

88  (2T031) 

172 

37,85 

68,41 

49,92 

51,13 

89  (2T131) 

99 

59,76 

58,25 

54,91 

57,14 

90  (2T231) 

303 

68,56 

65,32 

60,78 

64,6 

91  (2T331) 

31 

74,12 

61,74 

63,54 

65,03 

92  (2T332A) 

24 

66,91 

53,66 

55,83 

56,25 

93  (2T332B) 

17 

60,47 

46,64 

55,17 

56,05 

94  (2T431) 

124 

70,86 

53,53 

55,95 

63,78 

95  (2W031) 

778 

67,16 

67,49 

66,54 

67,44 

96  (2W131C) 

146 

62,98 

66,82 

61,07 

64,82 

97  (2W131E) 

287 

62,25 

64,71 

60,22 

64,69 

98  (2W131F) 

344 

61,7 

64,88 

61,2 

65,32 

99  (2W131 K) 

13 

72,46 

66,23 

64,76 

66,23 

100  (2W131L) 

12 

60,5 

67,75 

61,83 

66,75 

101  (2W131Z) 

14 

65,78 

67,42 

68,64 

72,21 

102  (2W231) 

146 

76,49 

66,56 

66,26 

72,03 

103  (3A031 ) 

909 

32,7 

68,1 

47,37 

47,21 

104(3C031) 

402 

57,07 

74,88 

76,49 

72,53 

105  (3C032) 

213 

78,52 

84,84 

92,2 

90,97 

106  (301 31 ) 

50 

34,32 

77,2 

46,54 

48,14 

107  (3C231) 

164 

66,76 

75,68 

75,19 

78,93 

108  (3C331) 

29 

50,2 

74,75 

71,2 

65,55 

109 (3E031) 

173 

54,57 

62,34 

58,5 

63,61 

110  (3E032) 

176 

74,35 

64,65 

65,44 

71,91 

111  (3E131) 

180 

57,96 

60,16 

55,66 

59,65 

112  (3E231) 

119 

66,43 

56,89 

58,47 

62,56 

113  (3E331 ) 

83 

71,54 

64,48 

63,65 

67,71 

114  (3E431) 

159 

69,32 

64,11 

62,83 

65,32 

1 1 5  (3E432) 

33 

67,6 

62,69 

60,27 

65,63 

116  (3E433) 

18 

46,16 

58,88 

56,88 

60,05 

117  (3E731) 

612 

57,41 

67,49 

63,89 

62,35 

118  (3E831) 

140 

80,3 

74,42 

81,07 

82,19 

119  (3M031) 

438 

37,24 

60,01 

52,08 

50,84 

120  (3P031 ) 

2382 

50,35 

65,86 

58,93 

58,6 

121  (3P032) 

769 

55,4 

66,4 

62,85 

62,48 

122 (3P032A) 

60 

62,83 

71,34 

68,26 

67,53 

123  (3P131) 

43 

76,55 

66,88 

62,76 

67,16 

124  (3S031) 

499 

35,66 

73,83 

50,25 

49,88 

125 (4A031 ) 

523 

39,52 

70,43 

60,01 

56,36 

126  (4A131) 

160 

42,65 

70,57 

58,14 

54,12 

127  (4A231) 

104 

68,01 

77,67 

80,06 

82,81 

128  (4B031) 

95 

51,54 

75,66 

69,04 

66,86 

129  (4C031) 

91 

45,14 

74,21 

70,76 

64,69 

130  (4D031) 

149 

42,59 

65,46 

60,09 

57,94 

131  (4E031) 

40 

43,77 

71,52 

65,77 

62,6 

132  (4F031) 

57 

45,64 

74,1 

65,71 

60,47 

133  (4H031) 

61 

52,67 

75,93 

70,52 

70,98 

134  (4J032) 

32 

54,56 

77,06 

74,12 

68,9 

135  (4M031) 

6 

43,83 

76,83 

70,33 

60 

136  (4N031) 

800 

46,82 

72,81 

63,51 

61,34 

37 


Job-ID  (AFSC) 

#  Assigned 

Mechanical 

Admin 

General 

Electronics 

137  (4N131) 

138 

47,95 

70,34 

64,55 

61,86 

138  (4P031) 

63 

45,36 

75,49 

65,53 

63,69 

139  (4R031) 

59 

52,25 

77,33 

65,76 

64,3 

1 40  (4T031 ) 

31 

51,51 

84,61 

77,48 

71,83 

141 (4T032) 

10 

67,2 

88,8 

87 

78,9 

142  (4V031 ) 

27 

43,96 

71,66 

67 

61,4 

143  (4Y031) 

204 

39,32 

68,76 

57,74 

56,01 

144(60031) 

66 

57,36 

81,21 

82,37 

76,36 

145 (6F031) 

205 

47,51 

75,64 

71 ,74 

65,99 

146  (6F032) 

272 

48,77 

75,11 

72.1 

66,13 

Total 

23578 

38 


ATTACHMENT  C:  ANALYSIS  OF  COMPOSITE  AVERAGES  FOR  DIFFERENT  TRADES 

(ORIGINAL  USAF  1996  ASSIGNMENT  -  PSYCHOMETRIC  MODEL) 

(DATA:  USAF  1996  (ASSIGNED  PER  Ktt 


Job -ID 

Rule 

Composite 

Minimum 

USAF 

Assignment 

PsyMod 

Assignment 

Difference 

#  Assigned 

Average 

#  Assigned 

Average 

1A231 

NA 

General 

55 

112 

73,23 

112 

88,28 

+15,04 

1A331 

NA 

Electronics 

67 

51 

81,62 

51 

95,03 

+13,4 

1A431 

NA 

General 

53 

57 

73,98 

57 

85,71 

+11,72 

1A531 

NA 

Electronics 

67 

27 

85,33 

27 

95,92 

+10,59 

1C031 

NA 

Admin 

45 

50 

70,12 

50 

87,3 

+17,17 

1C032 

NA 

Admin 

45 

195 

69,96 

195 

87,79 

+17,83 

1C131 

NA 

General 

53 

63 

74,88 

63 

85,49 

+10,61 

1C231 

NA 

General 

43 

1 

46 

1 

80 

+34 

1C331 

NA 

General 

48 

218 

62,54 

218 

82,93 

+20,39 

1C431 

NA 

General 

48 

48 

70,81 

48 

82,79 

+1 1 ,98 

1C531 

NA 

General 

53 

52 

70,59 

52 

85,69 

+15,09 

1N031 

NA 

General 

55 

105 

76,4 

105 

88,17 

+11,76 

1N131 

NA 

General 

64 

87 

78,09 

87 

97,75 

+19,65 

1N332A 

NA 

General 

69 

11 

88,63 

11 

98,81 

+10,18 

1N334G 

NA 

General 

69 

1 

74 

1 

99 

+25 

1N431 

NA 

General 

58 

49 

74,4 

49 

89,83 

+15,42 

1N531 

NA 

General 

69 

45 

82,88 

45 

98,84 

+15,96 

1N631 

NA 

General 

58 

24 

76,16 

24 

90,08 

+13,92 

1T131 

NA 

General 

30 

139 

55,1 

139 

40,33 

-14,78 

1W031 

AND 

General 

64 

290 

77,49 

290 

91,79 

+14,3 

1W031 

AND 

Electronics 

50 

290 

74,63 

290 

89,66 

+15,03 

2A031A 

NA 

Electronics 

67 

66 

78,31 

66 

95,48 

+17,17 

2A031B 

NA 

Electronics 

67 

72 

78,93 

72 

94,08 

+15,14 

2A131 

NA 

Electronics 

72 

77 

83,87 

8 

99 

+15,12 

2A132 

NA 

Electronics 

67 

3 

85,33 

3 

92,66 

+7,32 

2A133 

NA 

Electronics 

67 

16 

81,25 

16 

92,75 

+11,5 

2A134 

NA 

Electronics 

67 

7 

80,14 

7 

91,85 

+11,7 

2A137 

NA 

Electronics 

67 

91 

79,56 

91 

92,09 

+12,53 

2A331A 

NA 

Electronics 

67 

114 

79,27 

114 

90,11 

+10,84 

2A331B 

NA 

Electronics 

67 

156 

80,42 

156 

89,47 

+9,04 

2A331C 

NA 

Electronics 

67 

126 

77,58 

126 

87,38 

+9,79 

2A332A 

NA 

Electronics 

67 

162 

79,91 

162 

88,15 

+8,24 

2A332B 

NA 

Electronics 

67 

100 

79,55 

100 

87,96 

+8,4 

2A332C 

NA 

Electronics 

67 

81 

78,75 

81 

85,86 

+7,11 

2A333A 

NA 

Mechanical 

51 

354 

73,74 

354 

80,28 

+6,54 

2A333B 

NA 

Mechanical 

51 

317 

72,35 

317 

81,7 

+9,35 

2A333C 

NA 

Mechanical 

51 

17 

79 

17 

82,05 

+3,04 

2A333E 

NA 

Mechanical 

51 

80 

70,55 

80 

80,8 

+10,25 

2A333H 

NA 

Mechanical 

51 

32 

71,93 

32 

81,31 

+9,37 

2A431 

NA 

Electronics 

67 

121 

80,88 

121 

87,03 

+6,15 

2A432 

NA 

Electronics 

67 

126 

79,47 

126 

87,89 

+8,42 

2A433 

NA 

Electronics 

67 

81 

82,32 

80 

87,98 

+5,66 

2A531A 

NA 

Mechanical 

51 

209 

71,12 

209 

79,18 

+8,06 

2A531B 

NA 

Mechanical 

51 

80 

71,5 

80 

79,22 

+7,71 

2A531C 

NA 

Mechanical 

51 

306 

72,94 

306 

78,21 

+5,26 

2A531D 

NA 

Mechanical 

51 

101 

73,01 

101 

77,94 

+4,92 

2A531E 

NA 

Mechanical 

51 

121 

71,76 

121 

78,51 

+6,75 

2A531F 

NA 

Mechanical 

51 

83 

72,28 

83 

77,04 

+4,76 

2A531G 

NA 

Mechanical 

51 

166 

70,42 

166 

76,93 

+6,51 

2A531H 

NA 

Mechanical 

51 

116 

71,52 

116 

77,45 

+5,93 

2A532 

NA 

Mechanical 

57 

17 

79,29 

17 

92,35 

+13,05 

2A533A 

NA 

Electronics 

67 

42 

79,04 

41 

87,09 

+8,04 

2A533B 

NA 

Electronics 

67 

45 

80,68 

45 

87,46 

+6,77 

2A533C 

NA 

Electronics 

67 

18 

77,05 

18 

87,88 

+10,82 

2A631B 

NA 

Mechanical 

57 

85 

72,61 

85 

92,42 

+19,81 

2A631D 

NA 

Mechanical 

44 

96 

62,2 

96 

57,44 

-4,77 

2A631E 

NA 

Mechanical 

44 

89 

64,32 

89 

57,55 

-6.77 

2A632 

AND 

Mechanical 

51 

194 

72,73 

194 

76,24 

+3,5 

2A632 

AND 

Electronics 

33 

194 

68,62 

194 

78,44 

+9,81 

2A633 

NA 

Mechanical 

57 

92 

70,76 

92 

93,02 

+22,25 

2A634 

NA 

Mechanical 

51 

218 

67,16 

218 

76,77 

+9,61 

2A635 

NA 

Mechanical 

57 

294 

72,96 

294 

92,69 

+19,73 

2A636 

AND 

Mechanical 

45 

228 

70,12 

228 

92,41 

+22,28 

2A636 

AND 

Electronics 

60 

228 

75,97 

228 

89,94 

+13,97 

39 


Job-ID 

Rule 

Composite 

Minimum 

USAF  i 

Assignment 

PsyMod  i 

Assignment 

Difference 

#  Assigned 

Average 

#  Assigned 

Average 

2A731 

NA 

Mechanical 

51 

37 

74,62 

37 

75,83 

+1.2 

2A732 

NA 

General 

43 

105 

59,02 

105 

75,48 

+16,46 

2A733 

NA 

Mechanical 

51 

364 

69,5 

364 

75,14 

+5,64 

2A734 

NA 

Mechanical 

44 

71 

57,45 

71 

57,61 

+0,15 

2E031 

NA 

Electronics 

67 

38 

78,47 

38 

86,71 

+8,23 

2E131 

NA 

Electronics 

67 

128 

82,6 

124 

88,49 

+5,89 

2E132 

NA 

Electronics 

67 

20 

82,45 

19 

85,89 

+3,43 

2E133 

NA 

Electronics 

67 

97 

80,55 

93 

87,18 

+6,63 

2E134 

NA 

Electronics 

67 

25 

79,2 

25 

88,36 

+9,15 

2E231 

NA 

Electronics 

67 

81 

79,3 

79 

87,6 

+8,29 

2E231B 

NA 

Electronics 

67 

18 

86 

16 

87,18 

+1,18 

2E331 

NA 

Electronics 

67 

225 

79,91 

220 

86,45 

+6,54 

2E431 

NA 

Electronics 

67 

16 

80,62 

14 

87,57 

+6,94 

2E631 

NA 

Mechanical 

51 

86 

67,12 

86 

75,43 

+8,31 

2E632 

NA 

Mechanical 

51 

125 

69,72 

125 

73,9 

+4,18 

2E633 

NA 

Electronics 

46 

377 

67,76 

377 

65,72 

-2,05 

2E831 

NA 

Electronics 

67 

29 

81,48 

29 

86,51 

+5,03 

2F031 

AND 

Mechanical 

51 

456 

68,65 

456 

81 ,77 

+13,11 

2F031 

AND 

General 

39 

456 

60,7 

456 

82,02 

+21,31 

2M031A 

NA 

Electronics 

67 

108 

80,82 

104 

85,51 

+4,69 

2M031B 

NA 

Electronics 

67 

74 

81,41 

71 

83,11 

HZ 

2M032A 

NA 

Mechanical 

51 

121 

70,66 

121 

74,21 

+3,54 

2M033A 

NA 

Electronics 

50 

62 

69,72 

62 

71,85 

+2,12 

2P031 

NA 

Electronics 

67 

67 

81,37 

64 

86,93 

+5,56 

2R031 

NA 

General 

53 

45 

69,93 

45 

85,64 

+15,7 

2R131 

NA 

General 

43 

93 

56,91 

93 

75,5 

+18,59 

2S031 

OR 

Admin 

45 

932 

68,47 

932 

88,02 

+19,54 

2S031 

OR 

General 

43 

932 

47,82 

932 

63,68 

+15,86 

2T031 

NA 

Admin 

40 

172 

68,41 

172 

81,87 

+13,46 

2T131 

NA 

Mechanical 

44 

99 

59,76 

99 

57,74 

-2,02 

2T231 

AND 

Mechanical 

51 

303 

68,56 

303 

68,29 

-0,27 

2T231 

AND 

Admin 

32 

303 

65,32 

303 

89,02 

+23,7 

2T331 

NA 

Mechanical 

51 

31 

74,12 

31 

73,83 

-0.3 

2T332A 

NA 

Mechanical 

44 

24 

66,91 

24 

57,54 

-9,37 

2T332B 

NA 

Mechanical 

44 

17 

60,47 

17 

57,58 

-2,9 

2T431 

NA 

Mechanical 

51 

124 

70,86 

124 

74,2 

+3,34 

2W031 

OR 

Mechanical 

61 

778 

67,16 

778 

59,57 

-7,59 

2W031 

OR 

Electronics 

46 

778 

67,44 

778 

63,95 

-3,49 

2W131C 

OR 

Mechanical 

61 

146 

62,98 

146 

59,9 

-3,08 

2W131C 

OR 

Electronics 

46 

146 

64,82 

146 

64,21 

-0,61 

2W131E 

OR 

Mechanical 

61 

287 

62,25 

287 

59,07 

-3,18 

2W131E 

OR 

Electronics 

46 

287 

64,69 

287 

63,29 

-1 ,4 

2W131F 

OR 

Mechanical 

61 

344 

61,7 

344 

58,29 

-3,42 

2W131F 

OR 

Electronics 

46 

344 

65,32 

344 

62,92 

-2,4 

2W131K 

OR 

Mechanical 

61 

13 

72,46 

13 

58,84 

-13,62 

2W131K 

OR 

Electronics 

46 

13 

66,23 

13 

63 

-3,24 

2W131L 

OR 

Mechanical 

61 

12 

60,5 

12 

56,25 

-4,25 

2W131L 

OR 

Electronics 

46 

12 

66,75 

12 

62,16 

-4,6 

2W131Z 

OR 

Mechanical 

61 

14 

65,78 

14 

55 

-10,78 

2W131Z 

OR 

Electronics 

46 

14 

72,21 

14 

61 ,92 

-10,29 

2W231 

NA 

Mechanical 

61 

146 

76,49 

146 

97,46 

+20,97 

3A031 

NA 

Admin 

32 

909 

68,1 

909 

74,41 

+6,31 

3C031 

NA 

General 

60 

402 

76,49 

402 

94,17 

+17,68 

3C032 

NA 

General 

60 

213 

92,2 

213 

93,3 

+1,09 

3C131 

NA 

Admin 

45 

50 

77,2 

50 

87,04 

+9,84 

3C231 

NA 

Electronics 

67 

164 

78,93 

148 

84,95 

+6,01 

3C331 

NA 

General 

58 

29 

71,2 

29 

90,41 

+19,2 

3E031 

NA 

Electronics 

33 

173 

63,61 

173 

54,83 

-8,79 

3E032 

AND 

Mechanical 

57 

176 

74,35 

176 

83,78 

+9,43 

3E032 

AND 

Electronics 

43 

176 

71,91 

176 

83,84 

+1 1 ,93 

3E131 

OR 

Mechanical 

51 

180 

57,96 

180 

41,43 

-16,53 

3E131 

OR 

Electronics 

33 

180 

59,65 

180 

47,98 

-11,68 

3E231 

NA 

Mechanical 

44 

119 

66,43 

119 

58,23 

-8,21 

3E331 

NA 

Mechanical 

51 

83 

71,54 

83 

74,91 

+3,36 

3E431 

NA 

Mechanical 

51 

159 

69,32 

159 

73,51 

+4,19 

3E432 

NA 

Mechanical 

51 

33 

67,6 

33 

74,27 

+6,67 

3E433 

NA 

General 

39 

18 

56,88 

18 

65,11 

+8,22 

3E731 

NA 

General 

39 

612 

63,89 

612 

65,54 

+1,65 

3E831 

AND 

Mechanical 

61 

140 

80,3 

140 

94,88 

+14,57 

3E831 

AND 

General 

60 

140 

81,07 

140 

96 

+14,93 

40 


Job-1  D 

Rule 

Composite 

Minimum 

USAF 

Assignment 

PsyMod 

Assignment 

Difference 

#  Assigned 

Average 

#  Assigned 

Average 

3M031 

NA 

General 

30 

438 

52,08 

438 

40,98 

-11,11 

3P031 

NA  2 

General 

35 

2382 

58,93 

2382 

54,01 

-4,93 

3P032 

NA 

General 

35 

769 

62,85 

769 

54,57 

-8,29 

3P032A 

NA 

General 

35 

60 

68,26 

60 

52,86 

-15,41 

3P131 

NA 

Mechanical 

61 

43 

76,55 

43 

96,39 

+19,84 

3S031 

NA 

Admin 

45 

499 

73,83 

499 

87,21 

+13,37 

4A031 

NA 

General 

43 

523 

60,01 

523 

74,45 

+14,44 

4A131 

NA 

General 

43 

160 

58,14 

160 

73,73 

+15,59 

4A231 

NA 

Electronics 

67 

104 

82,81 

95 

85,65 

+2,84 

4B031 

NA 

General 

48 

95 

69,04 

95 

83,22 

+14,17 

4C031 

NA 

General 

53 

91 

70,76 

91 

86,35 

+15,58 

4D031 

NA 

General 

43 

149 

60,09 

149 

72,55 

+12,45 

4E031 

NA 

General 

43 

40 

65,77 

40 

72,09 

+6,32 

4F031 

NA 

General 

43 

57 

65,71 

57 

72,91 

fH 

4H031 

NA 

General 

43 

61 

70,52 

61 

72,06 

+1,54 

4J032 

NA 

General 

48 

32 

74,12 

32 

83,93 

+9,81 

4M031 

NA 

General 

43 

6 

70,33 

6 

71 

+0,67 

4N031 

NA 

General 

43 

800 

63,51 

800 

73,91 

+10,39 

4N131 

NA 

General 

43 

138 

64,55 

138 

76,3 

+1 1 ,75 

4P031 

NA 

General 

43 

63 

65,53 

63 

75,17 

+9,64 

4R031 

NA 

General 

43 

59 

65,76 

59 

75,28 

+9,51 

4T031 

NA 

General 

58 

31 

77,48 

31 

90,45 

+12,97 

4T032 

NA 

General 

43 

10 

87 

10 

74,8 

-12,21 

4V031 

NA 

General 

53 

27 

67 

27 

86,07 

+19,06 

4Y031 

NA 

General 

43 

204 

57,74 

204 

76,55 

+18,8 

6C031 

NA 

General 

70 

66 

82,37 

66 

98,78 

+16,4 

6F031 

NA 

Admin 

55 

205 

75,64 

205 

95,33 

+19,68 

6F032 

NA 

Admin 

55 

272 

75,11 

272 

95,07 

+19,95 

